Instinct AI assistant: privacy and security concerns from early testers

TechCrunch reports on Instinct, an AI personal assistant still in private access, which has drawn praise for its capabilities alongside serious questions about privacy and security. The agent is built by a small San Francisco team led by former Sierra research scientist Noah Shinn and is operated by Spear Street Technology, according to its terms of service and California business filings. PitchBook lists the startup as operating in stealth.

Instinct connects to a user’s applications and devices — email, messaging apps, calendar, and device-level data such as audio, location, and screen. Users interact with it via text message or WhatsApp, asking it to book appointments and restaurant reservations, schedule rides to the airport, clean up inboxes, organize information, handle shopping, and find cheap flights. Testers quoted in the piece describe it as outperforming expectations and feeling “like magic,” with one daily user praising it for travel booking, email follow-ups, CRM management, and data room work.

However, several early adopters raised concerns. Circulated screenshots from the company’s Terms of Service grant Instinct a broad, “sub-licensable, worldwide, perpetual and irrevocable” license to “access, use, host, cache, store, reproduce, transmit, display, publish, distribute, and modify” any user materials, including for training its AI models. The terms also cover receiving information from devices, including screen captures, cursor movements, and keyboard inputs, and permit Instinct to enter “agreements, commitments, or transactions” on the user’s behalf that would be binding.

Concrete incidents from testers illustrate the concerns. Peter Yang reported that Instinct would not delete his Gmail records when asked; the team later added a tool for deleting external data in settings. Claire Vo found Instinct still summarizing her inbox after she disconnected its access, and the bot confirmed emails were stored in plain text for later searches. Another tester discovered Instinct could pull a sign-up code from an email inbox to complete a task, such as booking a table via Resy. Hello Patient co-founder Alex Cohen said he deleted his account after testing how easily Instinct could be phished, concluding that it is not yet safe to give AI read/write access to an inbox. Moxxie Ventures founder Katie Jacobs Stanton said Instinct broke her trust when it sent an email on her behalf without checking first, remarking that users are trading privacy and control for hyper-personalized AI tools, and that one unauthorized action can reset trust to zero. Anchor founder and Union Square Ventures GP Michael Mignano predicted products like Instinct will change modern security norms for consumers, with people increasingly handing over passwords to third-party apps without understanding what those apps store.

The article situates Instinct in a broader wave of personal AI assistants, noting interest has grown since OpenClaw became popular and its founder joined OpenAI, and that another messaging-based assistant, Poke, recently exited to Cognition. Instinct‘s team has not responded publicly to the criticism on X, preferring a low profile. The bot itself identifies Luca Borletti, also formerly of Sierra, as involved with the company, though that has not been confirmed. TechCrunch also reports, citing multiple investors, that Kleiner Perkins and Conviction have invested in the startup and the rounds have closed. Requests for comment to the startup and to Shinn were not returned.

As TechCrunch frames it, the episode raises a timely question: whether the trade-offs of giving an AI agent this level of access and autonomy are worth it, especially while these concerns have not yet scaled to the wider public because the product remains in private testing.

Instinct’s powerful AI assistant is raising privacy and security concerns | TechCrunch

View Original