The LLM Revolution

🌈 Abstract

The article discusses the author's experience using text-to-speech (TTS) solutions and their exploration of using large language models (LLMs) for translation and other AI-powered services. It covers the following key points:

🙋 Q&A

[01] The author's experience with TTS solutions

1. What are the author's thoughts on the current TTS solutions they have used?

• The author has been using the flutter_tts package in their RSS reader Stratum, which hooks into the OS's native text-to-speech, making it free and available offline.
• However, the author finds the voices provided by this package to be of poor quality, especially on some platforms like Windows.
• The author is looking into using a premium TTS voice through an API, such as OpenAI's TTS system, which starts at $15 per million characters.

2. How does the cost of OpenAI's TTS system compare to their GPT-4o Mini?

• OpenAI's GPT-4o Mini is significantly cheaper, costing $0.15 per million input tokens, which is about 100 times cheaper than their TTS system.
• The author notes that TTS is billed by character, while GPT is billed by token, which is usually 2-4 characters.

3. What are the author's thoughts on the pricing difference between LLMs and TTS systems?

• The author finds it puzzling that LLMs like GPT are so much cheaper than TTS systems, and wonders why LLMs are priced so low.
• The author notes that ChatGPT even has a text-to-speech component, yet it is not as expensive as a dedicated TTS system.

[02] The author's experience with using LLMs for translation

1. What were the author's considerations when choosing a translation solution for their app Stratum?

• The author considered both traditional translation services like Google Translate and DeepL, as well as large language models like GPT and Gemini.
• The author heard good things about LLM-based translations but could not find empirical evidence that they are better than traditional translation services.
• The author ultimately decided to use Gemini 1.5 Pro, and later Gemini 1.5 Flash, due to its free tier and large context window.

2. How do the pricing models of Gemini and traditional translation services compare?

• Gemini 1.5 Pro costs $3.50 per million input tokens and $10.50 per million output tokens, while Gemini 1.5 Flash is significantly cheaper at $0.35/$1.05 per million tokens.
• In contrast, Google Translate and DeepL start at around $20 per million characters, which is significantly more than even Gemini 1.5 Pro.

3. What are the author's thoughts on the pricing of LLM-based translation services compared to traditional translation services?

• The author finds it puzzling that running an LLM is cheaper than a dedicated translation service, as it should be more expensive, not cheaper.
• The author suggests that the availability of open-source LLM frameworks, which can be tailored to specific use cases, is contributing to the lower costs of LLM-based translation services.

[03] The author's thoughts on the future of LLMs and their impact on the software industry

1. What are the author's concerns about the potential impact of LLMs on the software industry?

• The author expresses concern about a "race to the bottom" in the pricing of LLMs, which could lead to the demise of some companies and an "AI winter" if the AI market suddenly becomes unsustainable.
• The author worries about the potential for companies to suddenly pull out of the AI market or start charging market rates for their AI services, leaving customers dependent on those services without suitable alternatives.

2. How does the author view the potential of open-source LLM models like Llama?

• The author sees the availability of open-source LLM models like Llama as a positive development, as it imposes a price floor on LLMs and ensures that there will still be affordable AI tools available even if major companies leave the market.
• The author believes that LLMs are a transformative technology, similar to the steam engine, that will revolutionize the software industry, just as the steam engine revolutionized transportation.

3. What are the author's overall thoughts on the impact of LLMs on the software industry?

• The author initially had a more negative view on the disruptive impact of LLMs, but after further reflection, they see the potential for LLMs to be a positive force that can benefit smaller developers like themselves.
• The author believes that the LLM revolution, like the Industrial Revolution sparked by the steam engine, will lead to a resurgence and transformation of the software industry, making it more accessible and affordable for smaller players.