allm.link's answer
In 2024, a pivotal endeavor for both myself and my technical partner, Jack, has been the creation of a tool, accessible at https://allm.link, designed specifically for product managers and independent developers. The essence of this tool is encapsulated in its domain name: All LLM, One Link. It signifies a unified platform where users can harness the power of all large language models (LLMs) in one convenient location.
This project was born out of a shared frustration. In the latter half of 2024, Jack and I embarked on various AI tool development projects. To effectively navigate the landscape of LLMs, we found ourselves juggling numerous API accounts, each requiring its own payment solution. This was a cumbersome process, and we encountered setbacks, including account suspensions due to inexperience.
The process of prompt engineering was equally challenging. My workflow involved a chaotic array of open windows, each displaying a different LLM's API workbench or a markdown editor. This frantic switching was necessary to compare the performance of a single prompt across different models, as well as to experiment with different prompt formulations within the same model.
Once a satisfactory result was achieved, I had to use a token calculator (only provided by OpenAI), to estimate the cost of each interaction. If the cost was too high, I would need to find ways to compress and optimize the prompt. The token cost issue led us to explore alternative domestic models. However, the API workbenches of many Chinese models were rudimentary, sometimes non-existent. Ultimately, Jack built a temporary tool for me using our existing chatbot framework.
By late October, Jack proposed a significant overhaul of this tool. After a thorough discussion over coffee, we decided on two key objectives: first, to transform this tool into a paid service for others facing similar challenges; and second, to leverage my user experience as a blueprint to address the pain points of other product managers and developers.
Our design philosophy was primarily driven by the needs of an AI product manager. We recognized that a crucial aspect of being an effective AI product manager is to have experience with a wide range of models. Each model is trained with a unique set of values and biases, influenced by the data, labeling, reinforcement learning, and alignment processes. This is akin to raising a child, where the upbringing is inevitably shaped by the values and perspectives of the caregivers.
Consequently, different models react differently to the same input. For example, Claude is intelligent but cautious, and might react with disapproval if the conversation goes beyond a certain boundary. Grok, on the other hand, is much more permissive, reflecting the influence of Elon Musk's values. Pi is emotionally intelligent and empathetic, but it may not excel in logical reasoning or knowledge breadth.
Interacting with this diverse set of models presents its own challenges. Some models are restricted by their developers, while others are limited by geographical access or payment barriers. Some models require payment, while others lack the necessary API tools. Our goal is to overcome these obstacles and create an environment where all models can be accessed from a single point. We have made significant progress, integrating 18 model providers, including both Western and Eastern companies.
Moreover, we understand that newcomers to the world of LLMs often struggle with knowing what to ask. To address this, we plan to provide a curated list of prompts to help users get started.
Prompt engineering is a core skill for AI product managers. It is a cost-effective and efficient way to influence the behavior of large language models. Proficiency in prompt engineering can be achieved through innate talent or through extensive practice. This echoes the "10,000-hour rule," which emphasizes the importance of consistent practice.
Previously, I found the manual process of writing prompts to be arduous. To streamline this, I developed a prompt that generates prompts, a kind of "fighting fire with fire." We integrated this capability into the tool, allowing users to generate prompts and then modify them as needed. Users can save generated prompts and test them across different models. Ideally, multi-model testing would be done simultaneously. This feature, multi-model, multi-window comparison, is currently under development.
Cost awareness is also crucial for AI product managers. A common challenge with conversational AI products is that increased user engagement leads to higher costs. It is not ideal to penalize users for using a product more. Therefore, it's vital to plan carefully and be aware of cost boundaries during the product development phase.
It is advantageous to understand the pricing of each model API and the associated cost of each interaction during the design phase. The tool provides clear pricing information, with tooltips displaying each provider's API costs in USD. The tool also displays the token usage and cost per conversation in USD, with tooltips providing detailed information.
It is important to note that this tool is operated outside of mainland China. The pricing structure is based on actual usage, not fixed monthly fees. This is because fixed-price subscriptions often leave users feeling as though they have overpaid. We charge a small service fee on top of official API pricing to cover our operational costs, including servers, development, and payment processing.
For those who can access https://allm.link, we encourage you to register and try the tool. New users receive 1 million energy credits (equivalent to $1 USD), which should provide ample usage time. We hope you enjoy the tool.
SpectreIQ
Poe
Hugging Face
Typing Mind
HIX.AI
ModelDepot
