Large Language Models, or LLMs for short, can create human-like text and handle human-language complexity. Trained on vast amounts of data, LLMs can understand and produce text that’s contextually relevant across many different topics.
If you’re trying to create data aware LLM applications and talk to your data through your product or software then you’ll most likely run into LlangChain and/or LlamaIndex.
LangChain and LlamaIndex are both frameworks that allow you to ingest and query data using an LLM as an interface. Depending on your needs, one or both of these frameworks will make sense to leverage.
We’ll explore both in depth so that you can make an informed decision on the technologies you wish to implement in order to build your LLM application. It’s worth noting that both of these frameworks are constantly evolving and receive updates regularly.
What is LlamaIndex?
LlamaIndex (formerly GPT Index) is a framework for LLM applications to ingest, structure, and access private or domain-specific data.
As the name suggests, LlamaIndex is really focused on ingesting and structuring data based on index types, such as list or tree index. With LlamaIndex you can compose indices ontop of eachother to form complex data structures. As for querying, LlamaIndex is able to handle simple and more complex queries.
Below are the key features of LlamaIndex:
- Prompting: LlamaIndex uses prompting extensively to maximize its functionality.
- Document chunking: LlamaIndex divides documents into smaller chunks, leveraging LangChain’s textSplitter classes, which break down text to fit LLM's token limits. Customized chunking is also available.
- Graph index: The index can be a list, tree, or keyword table. There’s also the option to create an index from various other indexes, facilitating hierarchical document organization.
- Querying: When querying an index graph, two steps occur. First, relevant nodes related to the query are identified. Next, using these nodes, the response_synthesis module generates an answer. The determination of a node's relevance varies by index type.
- List index: Uses all nodes sequentially. In “embedding” mode, only the top similar nodes are used.
- Vector index: Uses embeddings for each document and retrieves only highly relevant nodes.
- Response synthesis: Multiple modes including 'Create and refine', 'Tree summarize', and 'Compact' determine how the response is created.
- Create and refine: In the default mode for a list index, nodes are processed sequentially. At each step, the LLM is prompted to refine the response based on the current node's information.
- Tree summarize: A tree is formed from selected nodes. The summarization prompt, seeded with the query, helps form parent nodes. The tree builds until a root node emerges, summarizing all node information.
- Compact: To economize, the synthesizer fills the prompt with as many nodes as possible without exceeding the LLM's token limit. If there are excess nodes, the synthesizer processes them in batches, refining the answer sequentially.
- Composability: LlamaIndex allows creating an index from other indexes. This is beneficial for searching across diverse data sources.
- Data connectors: LlamaIndex supports various data sources, like confluence pages or cloud-stored documents, with connectors available on LlamaHub.
- Query transformations: Techniques like HyDE and query decomposition refine or break down queries for better results.
- HyDe: HyDE (Hypothetical Document Embedding) prompts an LLM with a query to obtain a general answer without referencing specific documents. This answer, combined with the original query (if "include_original" is set to TRUE), helps retrieve relevant information from your documents. This method guides the query engine but can produce irrelevant answers if the initial query lacks context.
- Query decomposition: Queries can be undergo single or multi-step decomposition queries depending on the complexity.
- Node postprocessors: These refine the selected nodes. For instance, the
KeywordNodePostprocessor class further filters retrieved nodes based on keywords.
- Storage: Storage is crucial in this framework. It handles vectors, nodes, and the index. By default, data is stored in memory, except vectors which services like PineCone save in their databases. In-memory data can be saved to disk for later retrieval. Let's explore the available storage options:
- Document Stores: MongoDB and in-memory options available. MongoDocumentStore and SimpleDocumentStore handle storing document nodes either in a MongoDB server or in memory respectively.
- Index Stores: MongoIndexStore and SimpleIndexStore supports both MongoDB and in-memory for storing index metadata.
- Vector Stores: Supports in-memory and databases like PineCone. One thing to note is that hosted databases like PineCone are capable of very efficient complex calculations on vectors compared to in-memory databases such as Chroma.
To utilize LlamaIndex effectively, set up your storage preferences, and then generate a storage_context object for your indexes.
What is LangChain?
The framework’s focused on composition and modularity, and have a lot of individual components that can be used together or on their own such as models, chains and agents. LangChain also comes with use cases that show common ways to combine components.
Let’s dive into the various components of LangChain.
There are 3 kinds of models.
- LLMs: A large language model that takes in input via an interaction and returns a response as output.
- Chat models: Are similar to LLMs but specialized to work with message objects instead of pure text. Examples would be human message, system message and AI messages. These labels don’t do anything on their own, but help the system understand how it should respond in a conversation.
- Embedding models: These models are used to deliver text as a vector representation. A good use case for this is semantic search. LangChain uses two methods for embedding called embed_query and embed_document. This is because different LLMs use either one of these embedding methods.
Prompts are how users interact with LLMs. LangChain uses carefully designed prompts and prompt templates to take user query and data schemas in order to get the desired response.
There are four types of prompt templates that LangChain uses:
- LLM Prompt Templates: To enable dynamic prompt configuration and eliminate hardcoded values, Langchain offers an object rooted in Python's formatted strings. At present, Langchain supports Jinja, with plans to integrate more templating languages soon.
- Chat Prompt Templates: As previously described, we shift from using string objects to message objects here. Message objects offer a structured approach to conversational scenarios. Messages fall into three types: 1. HumanMessages, 2. AIMessages, and 3. SystemMessages.
The first two are straightforward in their naming. SystemMessages, however, don't originate from either AI or humans. They generally establish the chat's context. For instance, "You are a helpful AI that assists with resume screening" is a SystemMessage.
In essence, ChatPromptTemplates aren't merely based on strings as LLM prompts. They're grounded on MessageTemplates, which encompass HumanMessages, AIMessages, and SystemMessages.
- Example Selectors: LangChain offers the adaptability to decide the way in which you select input samples for a language model from a set of examples. There's a selector that functions based on input length, modulating the number of examples picked from your prompt according to the prompt's remaining length.
- Output Parsers: Inputs and prompts represent only one facet of LLMs. Occasionally, the manner in which the output is presented becomes importabnt, especially for subsequent operations. LangChain lets you use pre-designed output parsers, but you also have the freedom to make one tailored to your specific needs. For instance, there could be an output parser that translates the LLM response into a sequence of values separated by commas, suitable for saving in CSV format.
This tool retrieves information from documents based on a query. To build this system, we need a mechanism to load documents, create embedding vectors for them, and manage these vectors and documents. While LlamaIndex offers one approach, LangChain provides more granularity through its classes.
Document Loaders: This tool loads documents from various sources like HTML, PDF, Email, Git, and Notion using Unstructured, a pre-processing tool for unstructured data.
Text Splitters: Long documents can't be fully embedded into a vector model due to token size limits and the need for coherent chunks. Thus, splitting documents is essential. While LangChain offers splitters, creating a custom one for specific needs may be beneficial.
VectorStores: Databases to store embedding vectors, which enable semantic similarity searches. LangChain supports platforms like PineCone and Chroma.
Retrievers: Linked to VectorStore indexes, retrievers are designed for document retrieval, offering methods to determine similarity and the number of related documents. They are integral in chains that require retrieval.
Memory: While most interactions with LLMs aren't stored, memory is crucial for applications like chatbots. LangChain provides memory objects for tracking interactions. Examples include:
- ChatMessageHistory: Tracks previous interactions to provide context.
- ConversationBufferMemory: A simpler way to manage chat history.
- Saving History: Convert your message history into dictionary objects and save as formats like JSON or pickle.
Chains combine components, like language models and prompts, for specific outcomes. They can simplify inputs and provide detailed responses. They can be linked using classes like SequentialChain.
Agents and Tools
LLMs are bound by their training data. For real-time data, like weather updates, they need external tools. Chains might not be enough since they use every tool, irrespective of the query.
Agents decide which tool is relevant per query. Using agents involves loading tools, initializing the agent with them, and querying. For instance, an agent might use the OpenWeatherMap API and an LLM-math tool together.
Tools: LangChain offers standard tools, but users can create custom ones. Tool descriptions help agents decide which tool to use for a query. The tool's description is crucial for its effectiveness.
Similarities: LangChain vs LlamaIndex
LlamaIndex and LangChain have some overlap, specifically in the indexing and querying stages of working with LLMs.
Within this overlap, both frameworks do handle indexing and querying slightly differently.
Difference: LangChain vs LlamaIndex
While both frameworks provide indexing and querying capabilities, LangChain is broader and provides modules for tools, agents and chains.
Chains are a powerful concept in LLM development. With chains, the output from an interaction with your LLM can be your input for your next interaction.
For example, in a chatbot application a chain would represent a conversation between a user and the LLM. These steps are typically predefined or hardcoded into the applications.
Agents are similar to chains, but with decision-making powers. They can come up with steps on their own and decide what tools to use.
LlamaIndex’s primary use case is in going very deep into index and retrieval (querying).
Which one should you choose?
LangChain and LlamaIndex together provide standardization and interoperability when building LLM applications.
If you’re looking to get the most out of indexing and querying, and build really powerful search tools then LlamaIndex is most likely the stronger tool for the job, as it’s primarily focused on doing those two things well. LlamaIndex is also a bit easier to get started with, and has helpful getting started documentation.
To leverage multiple instances of ChatGPT, tools, interaction chains and agents that can autonomously work with tools, and provide them with memory then LlangChain is the way to go.
LangChain’s documentation is a bit more complex, though well-written, and might need a little more time to be productive with.
The community in LangChain has grown a lot faster than LlamaIndex, and so there will most likely be plenty of support around this framework along with content for common challenges in the future.
Finally, you can use both LangChain and LlamaIndex together and benefit the best of both worlds. This boils down to your needs when it comes to indexing and querying.
6 min read
Companies using machine learning understand that business growth requires continuous innovation.
Via machine learning, organizations are improving their businesses, both on the consumer level and in terms of business activity too.
To find out more about what machine learning can do, keep reading! You’ll learn all about 14 major companies using machine learning to change business for the better.
What Is Machine Learning?
Machine learning (ML) is a subset of artificial intelligence in which computers use data and algorithms to improve upon themselves.
Artificial intelligence (AI) itself describes any computer system that demonstrates human-like intelligence.
While most machines are set out to do a single task or a set of tasks, AI technology is more flexible.
Through machine learning, digital systems are constantly adapting to their surrounding environment.
Most businesses use artificial intelligence as a means of digital transformation.
By using AI, you can automate a number of tasks without so much as lifting a finger, from tracking lead behavior to sending mass emails at optimal times.
Consumers use AI technology too. And you have probably used AI more than you even realize.
For example, smart applications like Google Assistant and Siri are digital assistants that rely on artificial intelligence and machine learning to respond to user data and complete complex tasks.
In short, machine learning is more than capable of helping businesses and individuals alike meet their objectives.
Related reading: 8 Best Programming Languages for AI Development in 2022
15 Companies Using Machine Learning
Machine learning has been a popular concept in modern application development trends. Companies using machine learning have a variety of applications for this clever technology.
Here are some examples of major companies using machine learning:
Yelp hosts reviews from a large assortment of businesses all over the world.
The website can give locals and tourists recommendations of restaurants, bars, salons, and even dentists in their residing area.
With the use of machine learning, Yelp has fine-tuned image curation to provide users with more accurate photo captions and attributes.
Surely, if you take to reviews to learn whether or not a new spot in town is worth the hype, pictures are an essential part of the decision-making process. And obviously, accuracy is appreciated.
Pinterest is a social media service that’s a bit off-target from the norm. On Pinterest, users share ‘pins’ to help other users discover recipes, style inspiration, DIY projects, and other lifestyle ideas.
In 2015, Pinterest announced the acquisition of Kosei, a company that provides business-to-business (B2B) machine learning capabilities for its clients.
Now, machine learning is a fundamental part of almost everything that happens on Pinterest.
Spam moderation, content discovery, and even ad monetization take place with machine learning at the center.
Facebook is a notorious social media network that you’re likely well-familiar with. But did you know Facebook Messenger has an AI embedded within its infrastructure?
Or rather, businesses create chatbots that depend on the messenger’s internal software for its operations.
For example, you can message Domino’s Pizza for an app to take you through the ordering process.
Similarly, you can book a flight and hotel through Kayak with just a quick, easy chat.
You can even schedule a makeover at your local Sephora’s, all through Facebook Messenger!
Twitter is a social media service where users exchange information through concise, primarily text-based blurbs.
That said, not every tweet is welcome. Depending on who you are, you might find some tweets more relevant and/or entertaining than others.
Luckily, Twitter uses a machine learning algorithm to score tweets based on various metrics.
Then, Twitter curates your feed, making an educated guess about what you would like to see most.
Google is a proud user of neural networks. In humans, neural networks allow the brain to create relationships between large datasets.
Naturally, artificial intelligence can imitate this ability.
Right now, Google still uses classic algorithms for natural language processing (NLP), speech translation, and search ranking, among other things.
But they are deeply invested in researching and refining neural networks to enhance further development. Google dubs this project the DeepMind.
Baidu may not be a major company in the United States, but it is definitely one of the top companies using machine learning.
In fact, Baidu is one of the largest AI and internet companies in the world. It’s based in China.
Like Google, Baidu is also interested in neural networks. Through a project called Deep Voice, they plan to engineer human speech with genuinely human voices.
HubSpot has always put its best foot forward when it comes to investing in emerging technology.
As one of the more comprehensive marketing tools in the industry, machine learning is vital to how HubSpot works.
Marketing automation tools leverage artificial intelligence to optimize business activity all-around.
Predictive lead scoring or automatic record cleaning name only a few of the possibilities.
IBM, or International Business Machines, is one of the oldest technology companies that’s still alive and kicking.
But unlike other old companies, IBM continues to expand its technological resources. New and innovative technologies are regular addendums to the metaphorical IBM grocery list.
And thus, they continue to grow as a business.
One of IBM’s newer technologies is Watson, a machine learning tool that several hospitals and medical centers use to get treatment recommendations for different types of cancers.
But Watson can do more than that. The retail sector also uses Watson to assist shoppers. The hospitality industry uses Watson too. Truly, AI can do it all!
Salesforce and HubSpot compete in the same industry. Like HubSpot, Salesforce gives businesses the tools to scale their marketing efforts.
Much of this happens through the Salesforce CRM, or customer relationship management tool.
Salesforce software uses ML and AI to analyze customer engagement and work with you towards amplifying the customer experience.
Apple needs no introduction. But just in case, you should know that Apple is a multinational company that specializes in computer software and consumer electronics.
This means that not only is Apple behind the iOS that runs on iPhones but the iPhone itself. You can credit Apple with the Mac and its various operating systems as well.
Apple is also behind Siri. Siri is technically an AI machine in the form of a handy digital assistant.
She can send text messages, check your email, and answer random questions, among other tasks.
Although Intel does fall into the category of major companies using machine learning, Intel’s use of ML is unique.
Intel is a chip manufacturer, which sounds vague. But you very well could be using an Intel chip right now to power your computer’s core processor.
It could be the very reason you’re navigating 12 tabs at a time with glee.
That said, Intel has more than one niche. Their Nervana chips, for one, are designed for data center servers and utilize ML for major data processing.
Nervana chips are able to transfer 2.4 terabytes per second with very low latency, and apparently, machine learning is key to this prowess.
Microsoft is the backbone of numerous technologies, your Xbox and favorite word processor being only two examples.
As far as machine learning goes, that mostly has to do with their Maluuba acquisition. Maluuba is known for its impressive deep learning labs for NLP.
In practice, Microsoft might use this acquisition to empower its voice computing and voice search technologies.
So if you ever get annoyed with a few too many accidental Cortana pop-ups on your PC, expect your frustrations to be assuaged very soon.
Amazon is one of the largest retailers in the world.
Most people love Amazon because of its 2-day shipping, which makes immediate gratification an unhealthy but realistic expectation for consumer desires.
That aside, Amazon uses ML for many of its retail-oriented tasks, such as product recommendations, forecasting, data cleansing, and capacity planning.
Can you guess why Netflix is on the list of major companies using machine learning? Think suggestions.
Netflix is a highly-favored streaming service with more films and television shows than you can count readily available at its users’ fingertips.
But if that’s not reason enough to be enthusiastic, you don’t even have to do the work of figuring out what to watch.
That’s because Netflix's primary use of ML technology is to give user suggestions, catered specifically to their unique interests.
Machine learning can be instrumental in boosting productivity in your business or even revamping your latest software product.
Like most newer technologies, it may be a bit difficult to wrap your mind around the concept.
In this case, seeking out professionals to guide you in implementing ML within your organization is not a bad idea.
At Trio, we have expert developers at the ready for consultation and to fulfill any development need you have down the road.
Hire Trio developers today!
5 min read
Facial recognition is exactly what it sounds like. It refers to technology that can identify an individual using only their face.
This can occur via photo or video. But it’s worth noting that the technology is not always accurate, particularly towards people of color.
You’re probably most familiar with facial recognition applications in law enforcement.
Depending on the person, this can be a good or bad thing. Controversies around facial recognition often stem from privacy rights.
Still, it’s worth looking into just what facial recognition is capable of. There are many applications of facial recognition technology. And nowadays, OpenAI developers enable impressive advancements.
This article will explain six application of facial recognition technology:
- Automobile Security
- Access Control
The Growth of Facial Recognition Technology
One of the top tech trends in 2019 has been facial recognition, an intriguing technology that is starting to attract more business from various industries.
Its wide array of potential applications has made it easier for different industry verticals to adopt as the technology has become more mature. Typically associated with immigration and the security sector, today it's winning over healthcare, retail, and marketing many other markets.
Facial recognition software uses a mix of artificial intelligence (AI) and biometric technology to identify human faces through measuring nodal points, the distances between certain facial features. The software analyzes and compares patterns of a person's facial features to provide accurate verification of their identity.
The data from facial recognition has found its most practical use in security. Tech giants like Apple or Google utilize facial recognition in their mobile devices to verify their users’ identity, providing secure logins for devices.
Some other case uses involve government organizations who utilize the technology in places like airports to reduce identity theft, find missing people, and identify criminals.
The growth potential of face recognition technology is astonishing. By 2042, the global facial recognition technology market is projected to generate an estimated $15.4 billion – demonstrating a growth rate of 21.4% since 2016.
How Facial Recognition Technology Works
Biometric facial recognition applications are frequently used in consumer technology nowadays. For example, millions of users take part in the facial recognition capabilities of the iPhone X. The user’s face is identified by 30,000 separate infrared dots and adds an extra layer of security to the traditional identification methods.
The technology behind facial recognition varies from one application to another, but the basic mechanics of it, when used by the government for immigration and security, include four basic steps:
1. An image of your face is captured from a photo or video. To ensure more accurate matching, your face should be facing the camera, though the technology does allow some turns and can even pick out singular faces from crowds.
2. The software reads the geometry of your face. Key factors are the distance between your eyes and the distance from forehead to chin. The software identifies these facial features then creates your facial signature based on the profile they created.
3. Your facial signature is compared to a database of known faces. This data is powered by police databases and government security services.
4. If your facial signature is found to match one flagged in the database, it can be used to investigate and apprehend you.
Why Is Facial Recognition Important?
Facial recognition technology is most helpful to law enforcement officials trying to find criminals. Ironically, this is a controversial topic given the greater focus on privacy in the last 6-8 years.
But when facial recognition has relatively positive outcomes, it’s been known to find suspects in a matter of hours. And of course, when officers find the right guy for the crime, this is usually a good thing.
According to Spolin Law P.C, Facial recognition technology can prevent access to sensitive information. It can also serve as a record of who is accessing that information.
“As criminal defense lawyers at Spolin Law P.C., we use tech like this as evidence to support cases where someone may have been wrongfully accused. As with any type of crime, we often find there are mitigating circumstances that technology can support to help us achieve a fair outcome for all parties involved.”
Similarly, many small businesses have taken to using the same technology in order to empower themselves when faced with theft.
For personal use, you can shave off a few seconds unlocking your phone if you go with facial recognition technology rather than a fingerprint, pin, or password. Although experts have said that facial recognition is less secure than other security methods.
In general, the benefit of facial recognition technology is a sense of security and safety. And this is achieved through an ever-evolving technology.
What Are the Applications of Facial Recognition?
The advantages of security and safety have prompted many industries to implement facial recognition technology into their daily operations. Take a look at how and where this technology can be applied.
Although you probably don’t spend too much time thinking about them, every now and then you’ve probably seen an armored truck cruising around town. These trucks often carry important items, whether that’s intel or cash, and rely on facial recognition technology to prevent theft or even ensure that the driver’s eyes are on the road.
In another capacity, facial recognition technology is sometimes used by ride-sharing apps to confirm that a given passenger is who they say they are. Or alternatively, the same technology can guarantee that the passenger is approaching the right driver.
Outside of cars and smartphones, facial recognition can be used in the home to grant access to certain IoT devices in addition to entry into the home itself. As this technology becomes more and more advanced, people will feel better protected against home invasions and robberies.
Immigration offices exist as an extension to more well-known government segments. Facial recognition technology is used to enforce stricter border control, particularly when it comes to criminals and persons of interest who attempt to cross the border.
Other than federal and local security, facial recognition applications may exist most prominently in the education sector.
A growing number of schools already use cameras that utilize facial recognition software to identify students, staff, unauthorized individuals, and even behavior that could present a threat to safety. This is one of many new tech trends that are transforming education.
For schools using this technology, the main benefit they see is tracking student attendance as well as maintaining the security of their campus. Unfortunately, technology can be very biased and studies have shown evidence for the software to be banned.
Though the United States is behind in regards to using facial recognition for retail other countries like Japan have been doing so for quite some time. For example, vending machines in Japan can recommend drinks for the consumer by using facial recognition technology to approximate a customer’s gender and age.
Lately, it looks as if the USA is catching up. Amazon opened its first Amazon Go store in 2018. There is no check-out and the store depends entirely on sensors to figure out what a customer picks up and buys.
Given this, it might be safe to assume that Amazon has the brainpower to start mapping shopping behaviors in the near future once online shoppers are connected to offline faces.
Applications of facial recognition technology are used in hospitals, especially those working in assisted living. The software serves to keep track of everything that is going on within a hospital, ensuring patients are safe and the premise is secure.
If a patient wanders away from the care-giving facility with no identification, facial recognition can help quickly identify and find them quickly to prevent any harm from coming to them.
Concerns of Facial Recognition Technology
Facial recognition technology dates back to the 1960s. It has often drawn a large amount of criticism regarding the legality and ethics of its use.
This technology allows others to control the personal information of individuals and determine how it’s used. From this perspective, facial recognition may seem plainly immoral in practice.
Ownership of Data
While nobody can take away your physical face, digital images are another story. You can give up your right to ownership to your face when you sign a term of service agreement on a social network. Signing away the rights to your likeness allows images of you to be collected as data and sold.
As precise as facial recognition technology is, it can’t guarantee a 100% success rate. The possibility of false-positive results can lead to mistaken identities in crimes.
As mentioned, these inaccuracies harm people of color disproportionately. When people are falsely accused of crimes, the basic principles of law and order are put at risk.
When the government has the power to enter the private lives of its constituents, problems arise. Facial recognition begs the question of whether people should be entitled to privacy outside their homes.
For public safety and security purposes, facial recognition software has a lot of potential to stop crimes and control immigration/ But without proper regulation and oversight, there is potential for it to be misused. The debate surrounding this is constant and will progress as technology develops further.
While facial recognition is finding its way into new markets besides the government, the basic principles of the technology are the same and still pose the same risk to individual privacy.
Which Companies Are Using Facial Recognition?
Since Amazon’s consumer base is largely online, they may be in unfamiliar territory where facial recognition technology is concerned. But other companies are no stranger to this phenomenon. Here are some examples of companies using facial recognition.
The rumor is that Walmart is developing facial recognition technology that can determine a shopper’s mood. If a shopper is dissatisfied, Walmart hopes to respond to the issue accordingly and improve their customer service overall.
McDonald’s uses facial recognition in a similar way but the subject of the technology is different. In Japan, for instance, the technology is used to make sure that servers are maintaining a happy exterior.
Recently, ExxonMobile launched DriverDash. The app optimizes payments for fuel by using facial recognition to authorize transactions for drivers.
The beauty industry can certainly do some damage with facial recognition technology and they haven’t neglected to take the bait. At Covergirl, clients can choose a matching shade of foundation thanks to facial recognition technology.
As technology and innovation progress linearly, more markets will begin to open for this technology. It will be interesting to watch how it continues to disrupt various industry verticals.
Corporations and governments alike will continue to invest in these technologies. And here at Trio, we have already specialized in providing the best OpenAI developers to assist you with ethics and efficiency.
6 min read