Fun With AI Embeddings in Go

🌈 Abstract

The article discusses the author's experience of visiting San Francisco and the enthusiasm around AI and generative AI. It then describes the author's attempt to compare the lyrics of Taylor Swift and a heavy metal band using AI embeddings in the Go programming language. The article covers the process of scraping lyrics, generating embeddings, projecting the high-dimensional data into 2D and 3D visualizations, and analyzing the results.

🙋 Q&A

[01] The Motivation

1. What prompted the author to compare the lyrics of Taylor Swift and a heavy metal band? The author was curious about how Taylor Swift's lyrics compare to the lyrics of one of their favorite heavy metal bands, after a dinner conversation with friends where one of them summarized Taylor Swift's lyrics as "Taylor speaks the truth".

2. Why did the author choose to use AI embeddings to compare the lyrics? The author decided to use AI embeddings to compare the lyrics, as they felt it would be a good way to quantify and visualize the differences between the two artists, given the recent advancements in language models and text embeddings.

[02] The Task

1. What was the author's approach to generating the lyrics embeddings? The author used the OpenAI Ada model to generate embeddings for the individual song lyrics, as well as the average embeddings for each album. They also considered chunking the lyrics and calculating averages across the chunks, but opted for the simpler approach of using the full song strings.

2. How did the author project the high-dimensional embeddings into lower dimensions for visualization? The author used Principal Component Analysis (PCA) and t-SNE to project the high-dimensional embeddings into 2D and 3D visualizations, in order to make the data more interpretable.

[03] The Results

1. What insights did the author gain from the 2D and 3D projections of the song lyrics? The 2D PCA projections suggested that the lyrics of the two artists were fairly eclectic and diverse. The 3D t-SNE projections revealed more nuanced differences, showing that the lyrics were not as similar as the 2D projections had implied.

2. How did the author interpret the differences between the 2D and 3D visualizations? The author noted that the higher-dimensional 3D projections provided more context and information than the 2D projections, and that it's important to consider higher-dimensional data before simplifying to lower dimensions, as the lower-dimensional views can sometimes be misleading.

[04] Embeddings and their Potential

1. What are some of the interesting potential use cases for embeddings that the author discussed? The author discussed using embeddings for tasks like recommendation engines, data classification, network analysis, and reverse engineering. They also mentioned the potential of multimodal embeddings that can capture relationships between text, images, and audio.

2. How did the author emphasize the power and versatility of embeddings? The author stated that "Embeddings give you superpowers!" and that the ability to represent any data as numerical vectors opens up a wide range of mathematical operations and analysis techniques that can lead to novel insights and applications.