Everyone wants more bang for their buck when it comes to their business expenditures, and we want to ensure you have as many options as possible. Although you could certainly load full-weight fp16 models, it turns out that you may not actually need that level of precision, and it may

September 6th was a momentous day in large language model history, as Falcon-180 was released by the Technology Innovation Institute. To date, this is the single largest open-source LLM released to the public (edging out BLOOM-176b from 2022.) For quite some time, whether it was technical concerns or simply market

Generative AI makes developers lives much easier - but by how much? I have been learning German for the past year, and one of the things I thought would be personally useful would be to generate many conversations in German - via voice, which be extremely useful for me to

When the original LlaMA was released earlier in 2023 by Meta, it was only provided to the research community. With the next iteration, Meta in collaboration with Microsoft appears to have a change of heart and has released it as an open-source model for anyone to use. Here's

While some might prefer the simple text-based entry of an oobabooga interface, others might want something a little more robust. SillyTavern offers a number of additional features above and beyond most methods of interfacing with an LLM, and there's been some demand on getting it set up and

Hot off the heels of the 8192-token context SuperHOT model line, Panchovix has now released another set of models with an even higher context window, matching the 16384 token context possible in the latest version of text-generation-webui (Oobabooga). Such a large context window is going to vastly improve performacne in

Many LLMs (such as the classic Pygmalion 6b) are small enough that they can fit easily in almost any RunPod GPU offering. Others such as Guanaco 65B GPTQ are quantized which is a compression method. to reduce memory usage, meaning that you will be able to fit the model into

Esteemed contributor TheBloke has done it again, and textgen enjoyers everywhere now have another avenue to further increase their AI storytelling partner's retention of what is occurring during a scene. Available on his GitHub are quantizations of several well known models, including but not limited to the following:

As many blog entries in the past have been written on Oobabooga/text-generation-webui, we would be remiss if we failed to mention there was another much-loved frontend available for use on Runpod that may be of significant value to anyone interested in writing or roleplaying with an AI. KoboldAI comes

Since its inception, Oobabooga has had a hard upper limit of context of 2048 tokens for how much it can consider. Since this buffer includes everything in the Chat Settings panel including context, greeting, and any additional recent entries in the log, this can very quickly fill up to the

As fun as text generation is, there is regrettably a major limitation in that currently Oobabooga can only comprehend 2048 tokens worth of context, due to the exponential amount of compute required for each additional token considered. This context consists of everything provided on the Character tab, along with as