Instinct AI Assistant Sparks Privacy Worries Amid Powerful Capabilities
Instinct, a new private AI assistant, is drawing praise for its advanced features but also raising significant privacy and security alarms among early testers.
San Francisco-based Instinct, an AI personal assistant currently in private access, is generating significant buzz for its impressive capabilities, yet simultaneously sparking concerns regarding user privacy and data security. Developed by a small team led by former Sierra research scientist Noah Shinn and operated by Spear Street Technology, the AI agent aims to streamline daily tasks by connecting to users' applications and devices. Despite being hailed by testers as a "magical" and "exciting launch," the agent's broad access to personal data and its terms of service have prompted scrutiny from early adopters.
Instinct functions by integrating with a user's email, messaging apps, calendar, and device functionalities, including audio, location, and screen access. Users can interact with the AI via text messages or WhatsApp, requesting tasks such as scheduling appointments, managing inboxes, organizing information, and finding travel deals. While testers report that the AI often exceeds expectations, a growing number have voiced unease about the company's approach to safeguarding user information.
Concerns Over Data Access and Control
A central point of contention lies in Instinct's terms of service, which grant the company a broad, "perpetual and irrevocable" license to use user-submitted materials for training its AI models. These terms also permit Instinct to collect extensive device data, including screen captures, cursor movements, and keyboard inputs. Furthermore, the AI is authorized to enter into binding agreements and commitments on behalf of its users. Early tester Peter Yang reported that Instinct initially failed to delete his Gmail records upon request, a problem later addressed by the development team with a new data deletion tool. Claire Vo experienced a similar issue, with Instinct continuing to summarize her inbox even after access was revoked, confirming that the data was stored in plain text.
Security model questions also arose. One tester expressed concern after Instinct accessed their email inbox to retrieve a sign-up code for a restaurant reservation. Alex Cohen, co-founder of Hello Patient, cited a perceived vulnerability to phishing attacks as a reason for deleting his account. Similarly, Moxxie Ventures founder Katie Jacobs Stanton shared that her trust was broken when Instinct sent an email on her behalf without prior consent. Stanton summarized the broader dilemma: "We’re trading privacy and control for hyper-personalized AI tools (AI notetakers, personalized AI agents, etc) often without fully understanding the trade." She emphasized that the increasing power of such agents necessitates greater trust, noting that "Every successful action earns a little more trust. One unauthorized action can reset that trust to zero."
Michael Mignano, founder of Anchor and GP at Union Square Ventures, predicted that AI products like Instinct will reshape consumer security habits. "People will increasingly hand over passwords to 3p [third-party] apps, unaware of how or what they are storing for them," he stated.
The interest in advanced personal AI assistants has surged since the emergence of OpenClaw, another powerful AI assistant that led its founder to join OpenAI. The competitive landscape also includes other messaging-based assistants like Poke, which was recently acquired by Cognition. Despite the growing criticism and user concerns, the Instinct team has maintained a low profile, with no official response yet issued on platforms like X. Requests for comment from the startup and Noah Shinn have also gone unanswered. Investors, including Kleiner Perkins and Conviction, are reportedly involved, with funding rounds having closed. The company is currently operating in stealth mode, according to PitchBook data.