Create a secure chatbot that knows everything about what you’ve seen, said, or heard

I’ve built a tool that record your screen and microphone 24/7, while keeping the data 100% on your computer:

https://github.com/louis030195/screen-pipe

AI is very powerful nowadays but barely know anything about you or your company in particular.

Screenpipe is a library that allows you to gather all your life context and connect it to LLMs easily for:

• search (e.g. go beyond your limited human memory)
• automation (such as making actions on the web while you work, syncing company’s knowledge, etc.)
• etc.

Get started

You need to install a few dependencies:

# On Mac
brew install ffmpeg
# On Linux
sudo update
sudo apt install -y ffmpeg libasound2-dev libavcodec-dev libavformat-dev libavutil-dev

Also install Rust.

Clone screenpipe:

git clone https://github.com/louis030195/screen-pipe
cd screen-pipe

Build screenpipe:

# This runs a local SQLite DB + an API + screenshot, ocr, mic, stt, mp4 encoding
# if you are on mac, use "cargo build --release --features metal" to leverage M series acceleration
cargo build --release
./target/release/pipe


# or only stream audio + speech to text to stdout
./target/release/pipe-audio

# or only stream screenshots + ocr to stdout
./target/release/pipe-vision

# or only record mp4 videos + json containing ocr
./target/release/pipe-video

Update the variables in this directory (cp .env.example .env) :

# You must first activate a Billing Account here: https://platform.openai.com/account/billing/overview
# Then get your OpenAI API Key here: https://platform.openai.com/account/api-keys
OPENAI_API_KEY=XXXXXXXX

# Generate a random secret: https://generate-secret.vercel.app/32 or `openssl rand -base64 32`
AUTH_SECRET=XXXXXXXX

# Instructions to create kv database here: https://vercel.com/docs/storage/vercel-kv/quickstart and 
KV_URL=XXXXXXXX
KV_REST_API_URL=XXXXXXXX
KV_REST_API_TOKEN=XXXXXXXX
KV_REST_API_READ_ONLY_TOKEN=XXXXXXXX

And run the web app:

cd examples/ts/vercel-ai-chatbot
pnpm i # or npm i
pnpm dev # or npm run dev

Enjoy!

If you like screenpipe, please drop a star:

https://github.com/louis030195/screen-pipe

Struggle to get it running? I’ll install it with you in a 15 min call.

Louis

x.com