NVIDIA has released a chatbot that runs directly on a user's PC
NVIDIA
NVIDIA has unveiled Chat with RTX, a local chatbot that runs on the user's personal computer.
Here's What We Know
Chat with RTX allows personal data to be attached from a hard drive for use with open language models like Mistral or Llama 2. Based on these files, the chatbot can, for example, build a CV or answer questions from the user.
The programme supports a variety of data formats, including .txt, .pdf, .doc/.docx and .xml. According to NVIDIA, it takes seconds to index and download the necessary files. Contextual information from the documents is then integrated to deliver more relevant responses.
Chat with RTX also supports integration with YouTube videos and playlists. The user can provide a link to a video and the chatbot will analyse the content to draw on this knowledge when searching for answers.
Chat with RTX runs locally without internet access, which guarantees data privacy and takes into account the personal context of the PC owner. This differentiates it from cloud-based chatbots that relay requests to remote servers.
For now, this is a demo version of the product, so the developers have warned of bugs. Chat with RTX requires a GeForce RTX 30-series or higher graphics card and a minimum of 8GB of VRAM to run.