GPTs Are the Wrong Tool for Building AI Apps

🌈 Abstract

The article discusses the author's perspective on OpenAI's custom GPTs (Generative Pre-trained Transformers), highlighting their concerns and critiques about this offering.

🙋 Q&A

[01] Reasons for Being Underwhelmed by Custom GPTs

1. What are the author's main criticisms of the name "GPT" used for OpenAI's custom models?

• The name "GPT" is confusing, as it is already used for the larger GPT-x language models. The custom models are not sophisticated enough to warrant the "GPT" name.
• The name implies the creation of a new version of the LLM model, when in reality the custom GPTs are just adding a bit of prompt grounding.

2. What are the author's concerns about the high token usage and cost of running custom GPTs?

• Custom GPTs deplete the author's paid tier daily allowance of GPT-4 usage extremely quickly, even when avoiding tools like the GPT Builder.
• The author is concerned about their potential GPT users using up all their tokens in a short time when using the custom GPT.
• The lack of support for running custom GPTs under the cheaper GPT-3.5 model exacerbates this issue.

3. Why does the author consider using LLMs to handle application logic as "tragically stupid" from a software architecture perspective?

• It is computationally wasteful to load application logic into the LLM's context window, encode it, transform it, execute on a neural net, and decode it, when a conventional software approach would be much more efficient.
• The delay caused by this process is unacceptable for basic UI and application functionality, compared to when the LLM is used for tasks that can only be done with an LLM.
• The unreliable nature of generative AI means there is a risk of the application logic being "hallucinated" incorrectly, which is unacceptable for certain tasks.

[02] Positive Aspects of Custom GPTs

1. What features of custom GPTs does the author find interesting or useful?

• The potential of using "actions" for retrieval-augmented-generation (RAG) within custom GPTs.
• The convenience of reusing prompt groundings that are set up as custom GPTs in the ChatGPT portal.
• The ability to create shareable links to custom GPTs, allowing the release of OpenAI-based software without hosting costs.

2. Why does the author consider the share-ability of custom GPTs to be the "very best feature"?

• It allows the author to release LLM-based apps without having to worry about hosting costs, as users already have OpenAI accounts (many of them paid) to cover the hosting costs.
• From OpenAI's perspective, the author's app should serve as an incentive for people to get and maintain their OpenAI accounts.

3. What alternative solution does the author suggest to the issue of hosting costs for LLM-based apps?

• The author suggests that OpenAI could provide an OAuth-based API, similar to how other companies allow users to authenticate and authorize access to their APIs. This would eliminate the need for users to enter API keys, which the author considers cumbersome and potentially insecure.

[03] Conclusion

1. What is the author's overall conclusion about the usefulness of custom GPTs?

• The author concludes that custom GPTs are decent for personal use in some cases, but a poor choice for building serious apps or even just demos to share with other people.
• The author suggests that playing with custom GPTs for a bit is worth it, but they would not choose that platform to invest more than a Sunday afternoon learning.

2. What alternative approach is the author interested in exploring instead of custom GPTs?

• The author is interested in exploring the use of local LLMs, such as Ollama and WebLLM, as a way to create zero-hosting-cost LLM apps. The author appreciates the increased control and privacy offered by this approach, even though the models may not be as capable as GPT-4.