In the fast-paced world of artificial intelligence development, one of the key factors that can determine the success of enterprises is the availability of large language models (LLMs). The regional availability of these models can provide a significant competitive advantage, allowing faster access to innovation. However, many organizations face challenges when models are not yet available in their tech stack’s location. This can be due to resource constraints, western-centric bias, or multilingual barriers, forcing them to wait and potentially fall behind in the race for AI advancement.
To address this critical obstacle, Snowflake recently announced the general availability of cross-region inference. This feature allows developers to seamlessly process requests on Cortex AI in a different region, even if the desired model is not yet available in their source region. With a simple setting, new LLMs can be integrated as soon as they are released, enabling organizations to stay ahead in the AI game. This breakthrough innovation opens up opportunities for organizations to utilize LLMs privately and securely in regions such as the U.S., EU, and Asia Pacific and Japan (APJ), without incurring additional egress charges.
To leverage the power of cross-region inference, developers must first enable this feature on Cortex AI. By specifying the regions for inference and setting the necessary parameters, data traversal between regions can be facilitated. If both regions operate on Amazon Web Services (AWS), data will traverse securely within the global network with automatic encryption at the physical layer. In cases where regions are on different cloud providers, traffic will be encrypted and transported via mutual transport layer security (MTLS) through the public internet. It is important to note that inputs, outputs, and service-generated prompts are not stored or cached, ensuring secure and efficient processing of inferences.
Developers can configure the regions for inference processing based on their requirements. By setting account-level parameters, Cortex AI will automatically select a region for processing when a requested LLM is not available in the source region. For example, if a parameter is set to “AWS_US,” the inference can be processed in U.S. east or west regions. Similarly, setting a value to “AWS_EU” will route the inference to central EU or Asia Pacific northeast. Currently, target regions can only be configured to be in AWS; requests in Azure or Google Cloud will still process in AWS. This flexibility allows organizations to optimize their inference processing based on regional availability and requirements.
Snowflake Arctic is a prime example of how cross-region inference can streamline inference processing. In a scenario where Arctic is not available in AWS U.S. east, Cortex AI identifies this limitation and routes the request to AWS U.S. west 2 for processing. The response is then sent back to the source region seamlessly, all achieved with just one line of code. Users are charged credits based on the consumption of the LLM in the source region, ensuring cost-effectiveness and efficient utilization of resources.
Cross-region inference is a game-changer in the realm of large language models, overcoming regional availability challenges and enabling organizations to harness the power of AI innovation. Snowflake’s innovative solution provides a seamless and secure way to integrate new LLMs, ensuring that enterprises can stay ahead in the competitive AI landscape. By leveraging the capabilities of cross-region inference on Cortex AI, organizations can unlock new possibilities and drive transformative advancements in artificial intelligence.
Leave a Reply