Elon Musk's secretive artificial intelligence startup xAI has officially announced its first product - an AI assistant named Grok. Designed to answer questions with humor and wit, Grok aims to be an AI guide modeled after the Hitchhiker's Guide to the Galaxy.
Similar to other popular AI assistants like OpenAI's ChatGPT, Anthropic's Claude and Google Bard, Grok is designed to be adept at information retrieval, creative writing and coding. Powered by the Grok-1 language model, the current version has a context length of 8,192 context tokens.
xAI stresses that Grok is still an early beta product after just two months of training. However, it notes that one of Grok's advantages is its "real-time knowledge of the world" through integration with Musk's X platform (formerly Twitter). xAI also boasts Grok will "answer spicy questions rejected by most other AI systems."
Grok-1 boasts significant capabilities despite its "modest" model size (apparently billion parameters). On mathematical reasoning benchmarks like GSM8k and MMLU, as well as HumanEval code completion, Grok-1 surpasses all other models in its compute class. xAI claims Grok-1 displays "rapid progress" given its comparatively limited training data and compute.
Beyond standardized reasoning benchmarks, xAI evaluated Grok-1 on the 2023 Hungarian national high school math exam. This was to eliminate the possibility that questions in the evaluation benchmarks were part of Grok's training data. xAI says it served as a "real-life" test on novel data, since the exam was published after Grok-1's training data collection.
Without explicit tuning for this exam, Grok-1 passed with a 59% C grade at temperature 0.1 using the same prompt as Claude-2 and GPT-4. Claude-2 achieved a similar 55% C grade, while GPT-4 scored a 68% B.
xAI founding member Toby Pohlen shared some videos showcasing Grok's user interface and unique features.
Concurrent Conversations
One video shows how Grok allows users to conduct multiple AI conversations simultaneously and switch between them, enabling efficient multi-tasking.
Conversation Branching
Another video displays Grok's ability to branch conversations and explore variations of the initial question. Users can navigate between branches in a response tree to compare Grok's answers. This facilitates recursively probing Grok's knowledge.
Conversation Steering
Users can open up Grok's responses in a markdown editor, directly edit the text, save it externally, and then continue the conversation with the edited version. This allows human users to overwrite any part of Grok's response and have the AI continue as if that was its original output.
The editing works in tandem with Grok's branching features, so users can edit responses down different conversational branches and Grok will maintain context with the edited text. Giving humans direct editing access to alter Grok's responses before continuing could allow iteratively "steering" the AI or correcting any erroneous text.
xAI says it utilizes a custom Kubernetes stack in Rust focused on maximizing uptime and efficiency at scale. The company is now expanding infrastructure to support future jumps in model size. The company also highlighted key research directions like scalable human oversight, adversarial robustness, and multimodal capabilities.
Elon Musk has said that Grok will be offered as part of the X Premium+ subscription for $16 per month. He further noted that Grok will be both a built-in feature on the X platform as well as a standalone application. However, it is still unclear if this means that non-X users will be able to access Grok. Eagled-eyed readers may have noticed that the videos shared by xAI's Toby Pohlen depict Grok running as a native Mac desktop application.
Grok is initially being offered via a waitlist to select users in the United States. Wider release plans are unclear. But xAI has an "exciting roadmap" to roll out new features in coming months as user feedback helps refine Grok's capabilities.
The unconventional persona xAI has built for Grok reflects Musk's contrarian outlook. With rivals like Claude racing to release advanced AI assistants, Grok may aim to differentiate itself through its irreverent tone and promised rejection of political correctness.
The private beta release of Grok, merely four months after the inception of xAI, is likely to raise some eyebrows, particularly in the context of Elon Musk's vocal stance on AI safety. Musk has consistently advocated for caution in AI development to prevent potentially catastrophic outcomes. This rapid turnaround seems somewhat at odds with his calls for a slow and careful approach to AI advancement.
For comparison, OpenAI says it invested six months in enhancing the safety and alignment of GPT-4 using methods like Reinforcement Learning from Human Feedback before it began a public rollout. The limited information available about the safety precautions implemented in Grok's development and release begs questions about how xAI is ensuring the responsible and safe use of its AI model. Additionally, it remains to be seen how the company plans to distinguish itself from the practices of other companies that Musk has been critical of in the past.
More transparency and guardrails would be prudent given Musk's own spirited criticism of other firms' AI ethics. After all, the long-term interests of humanity should outweigh short-term technological showmanship.