Using UI with AI

If your web or mobile app has multiple user interface (UI) commands, (such as log in, register, search, show products, change user settings), users might struggle to know exactly where to click. The UI would be much more user-friendly if an AI could interpret user speech and convert it into commands that can be handled by the backend. Today’s AI is robust enough to map different phrases that mean the same thing to a single command. For example, a user might say "register" or "create a new account," and both can be mapped to the command "sign_up." The AI can understand both English and Turkish, for example "bana yeni bir kullanıcı oluştur" correctly maps to "sign_up". Here is a demo in Python:

When you use an API, such as OpenAI, the main disadvantage is that you must pay for every API call. Therefore, using voice commands to control the UI should be limited to paying customers, and there should be rate limits in place to keep costs under control. You might use open-source models like LLaMA to run the AI on your own server, but that would require better computational and memory resources than you currently have.

17.02.2025: Open source AI models like DeepSeek open the door to self hosted AI. You will need a powerful server with lots of RAM and GPU. Such servers might cost more than AI API calls if you use a cloud server. One solution might be to have your own physical server to run the AI model and use the cloud server for the web app, which makes API calls to the AI on your server.