Open Interpreter - Introducing Local III

🌈 Abstract

The article discusses the release of Local III, an update to the Open Interpreter project that aims to provide personal, private access to machine intelligence. The update includes features like a local model explorer, integrations with inference engines, custom profiles for open models, and a free hosted opt-in model for training an open-source language model. The article also highlights the community contributions and the project's goal of ensuring collective freedom and private, local access to powerful AI agents.

🙋 Q&A

[01] Local III Update

1. What are the key features introduced in the Local III update?

• Easy-to-use local model explorer
• Deep integrations with inference engines like Ollama
• Custom profiles for open models like Llama3, Moondream, and Codestral
• Suite of settings to make offline code interpretation more reliable
• Free, hosted, opt-in model via interpreter --model i
• Easier to use local models with interactive setup to select inference provider, model, and download new models

2. How does Local III enable local vision and OS mode support?

• Images sent to local models are rendered as a description generated by Moondream, a tiny vision model, and the model also receives OCR extracted from the image.
• Local III enables experimental local OS mode support, where Open Interpreter can control the mouse, keyboard, and see the screen, allowing the language model to interact with the computer by clicking icons identified by the open-source Point model.

3. What is the goal of the Open Interpreter project with the release of Local III?

• To ensure a destiny for computers where the intelligence age is controlled by the people, not an oligopoly of language model providers.
• To provide a balancing force against the control of language models by a few providers and develop a response to ensure collective freedom and private, local access to powerful AI agents.

[02] Community Contributions

1. What are some of the notable community contributions highlighted in the article?

• Fix Jupyter logging on shutdown by @tyfiero
• Check for GPU or MPS availability before using CPU by @jcp
• Add DevContainer Support by @weihongliang233
• Updated to address comment regarding pip installer not working by @MartinLBeacham
• Added 'py' alias for Python/JupyterLanguage by @CyanideByte
• Fixed broken link to setup by @Sandeepsuresh1998
• Added Ruby support by @bars0um
• Fixed task completion message looping by @CyanideByte
• Generated history conversation filenames in Chinese properly by @Steve235lab
• Displayed link to docs with unrecognized flags by @benxu3
• Added offline doc how-to in README by @dheavy
• Fixed bug by @imapersonman
• Added argument minor refactor by @MikeBirdTech
• Added flag reset to re-import computer instance by @meawal
• Modified API key storage user recommendation by @rustom
• Fixed profile.disable_telemetry not working by @LucienShui
• Fixed default variable issue by @tyfiero
• Removed pydantic warnings by @imapersonman
• Added multiple display support by @Amazingct
• Updated litellm to fix pydantic warning by @CyanideByte
• Fixed computer.calendar dates issue by @supersational
• Optimized rendering of dynamic messages by @kooroshkz
• Ignored empty messages by @CyanideByte
• Fixed optional import crash and error by @CyanideByte
• Added function to contribute conversations by @tyfiero
• Added Ollama with Llama3 as default by @tyfiero
• Segmented default.yaml into sections for clarity by @zdaar
• Removed config from docs by @MikeBirdTech
• Fixed Llama3 backtick hallucination in code blocks by @CyanideByte
• Fixed %% magic command by @Notnaton
• Refined documentation formatting and style by @RateteApple
• Bumped version of tiktoken by @minamorl
• Updated llm.py to use litellm.support_function_calling() by @Notnaton
• Updated local profile to not use function calling by @Notnaton
• Added local OS profile for local OS control by @MikeBirdTech
• Updated litellm for namespace conflict warning fix by @CyanideByte
• Added Spanish readme translation by @palnever
• Updated README_ZH.md by @KPCOFGS
• Added batch, bat aliases for shell language by @CyanideByte
• Fixed Llama 3 code hallucination by @Notnaton
• Fixed Linux installer by @okineadev
• Updated installation scripts by @okineadev
• Fixed typos by @RainRat