Generative AI workflows and hacks 2024
March 23, 2024 — October 4, 2024
I will try to synthesise LLM research elsewhere. This is where I keep ephemeral notes and links.
1 Chat API GUIs
Want a decent UX while using your own (possibly better-secured) API endpoint? Here are some options: NB, they all claim to keep your API key secret, but you should verify this yourself. Stealing API keys, or even better API keys plus confidential LLM queries, is a high-value target for attackers.
LibreChat
Enhanced ChatGPT Clone: Features Anthropic, AWS, OpenAI, Assistants API, Azure, Groq, o1, GPT-4o, Mistral, OpenRouter, Vertex AI, Gemini, Artifacts, AI model switching, message search, langchain, DALL-E-3, ChatGPT Plugins, OpenAI Functions, Secure Multi-User System, Presets, completely open-source for self-hosting. Actively in public development.
Dependency-heavy, so it prefers to be installed by Docker.
- danny-avila/LibreChat
- quick start
- weird overrides are necessary to get access to the custom endpoints which was my whole point for being here. Make sure that the
librechat.yamlconfig is active. Once it is, configuration is flexible; e.g.
huggingface/chat-ui: Open source codebase powering the HuggingChat app
Seems to be more oriented to local LLMs, but handles remote ones too, e.g.
Open WebUI (Formerly Ollama WebUI)
Light install, but sensitive python version requirements. I had to use pyenv to get it to work.
It does not seem to support multiple OpenAI Azure endpoints, which means being stuck to a specific model.
ChatGPTNextWeb/ChatGPT-Next-Web (Free. This one seems to work well) If you trust the maintainers, you can use the install deployed on the web.
Lovely UX. Error messages for custom endpoints are not great.
lobehub/lobe-chat: Much like ChatGPT web, lovely UX. Also not great at endpoint switching.
MindMac (expensive native macOS app)
ai-genie/chatgpt-vscode: Your best AI pair programmer in VS Code A functional client. If all you care about is code generation, this is a good choice. If you want other features, such as flexibility, easy switching of endpoints etc, no good.
The next two I have heard recommended but have not tested:
2 Multi-agent LLMS
Autogen
AutoGen Studio: Interactively Explore Multi-Agent Workflows | AutoGen
RUCAIBox/LLMSurvey: The official GitHub page for the survey paper A Survey of Large Language Models. / [2303.18223] A Survey of Large Language Models
3 Misc
Calculating Sunflower Oil Production (ChatGPT psychosis)
-
Large language models (LLMs) are emerging as a transformative technology, enabling developers to build applications that they previously could not. But using these LLMs in isolation is often not enough to create a truly powerful app - the real power comes when you are able to combine them with other sources of computation or knowledge.
This library is aimed at assisting in the development of those types of applications.
Prompt Airlines - AI CTF by Wiz
Your goal is to manipulate the customer service AI chatbot to get a free airline ticket
4 Running locally
First, get a fancy fast computer. Then: