نشر نماذج اللغات الصغيرة على الحافة باستخدام AWS IoT Greengrass ووكلاء Strands

Modern manufacturers and various other industries are increasingly seeking ways to leverage AI-powered solutions for real-time data processing and intelligent decision-making at the edge. This blog post details how to deploy Small Language Models (SLMs) at scale using AWS IoT Greengrass and Strands Agents, enabling context-aware insights directly on edge devices while maintaining security and performance. Unlike their larger counterparts, SLMs are designed to fit within the resource constraints of industrial environments, making them ideal for scenarios where reliability and low latency are critical.

Key Benefits and Use Cases

• Real-time Insights: Enables operators to query equipment status, interpret telemetry, and access documentation instantly without cloud connectivity.
• Hybrid Approach: Combines local inference for immediate responses with cloud resources for complex analytics, multi-site optimization, and model retraining.
• Versatile Applications: Applicable across industries, including automotive (voice commands), energy (SCADA data processing), gaming (companion AI), and higher education (personalized learning).

Solution Overview

The architecture leverages AWS IoT Greengrass to deploy and manage SLMs on edge devices, complemented by Strands Agents for local agent capabilities. Key AWS services involved include:

• AWS IoT Greengrass: Deploys, manages, and monitors device software at the edge.
• AWS IoT Core: Connects IoT devices securely to the AWS cloud.
• Amazon S3: Stores and retrieves SLM model files in GGUF format.
• Strands Agents: Provides a Python framework for running multi-agent systems, orchestrating local and cloud-based inference.

The solution utilizes an OPC-UA simulator to represent a factory environment with an oven and conveyor belt, along with maintenance runbooks for industrial data. The workflow proceeds as follows:

1. A model file in GGUF format is uploaded to an Amazon S3 bucket accessible to AWS IoT Greengrass devices.
2. Devices receive a file download job handled by the S3FileDownloader component, which efficiently manages large file transfers.
3. The model is loaded into Ollama, a software that runs inference on GGUF model files, when first called by Strands Agents component.
4. A user query is sent to the local agent via a device-specific MQTT topic in AWS IoT Core.
5. The Strands Agents SDK uses its Orchestrator Agent to understand the query, determine relevant information sources, and delegate to specialized agents like Documentation Agent and OPC-UA Agent.
6. Documentation Agent retrieves information from documents, and OPC-UA Agent queries the OPC-UA server for real-time or historical machine data.
7. The Orchestrator Agent aggregates the information and publishes a response to a device-specific response topic.

• Strands Agents SDK: Facilitates interaction with locally deployed models (Ollama) and offers the flexibility to switch to cloud-based models like those in Amazon Bedrock when available.
• IAM Role and IoT Certificate: Enables secure access to S3 resources and MQTT communication with AWS IoT Core.
• Logging: Component operation is logged locally, with optional integration with AWS CloudWatch for cloud-based monitoring.

Deployment Walkthrough

The blog post provides a detailed walkthrough including the following steps:

1. Clone the Strands Agents repository from GitHub.
2. Use the Greengrass Development Kit (GDK) CLI to build and publish the component. Configuration of the gdk-config.json file (region and bucket) is a requirement.
3. Deploy the component to the edge device via the AWS IoT Greengrass console.

For testing in an Amazon EC2 instance, a CloudFormation template is provided to deploy a GPU instance with pre-installed software and necessary Greengrass resources.

Model File Management

Downloading the SLM file is a critical step:

• The model file, in GGUF format, must be placed under /tmp/destination/ on the edge device and named model.gguf.
• Alternatively, the S3FileDownloader component can be used to download the model file from an S3 bucket, offering automatic retry and resume capabilities for unreliable connectivity scenarios. A sample payload for the S3FileDownloader component is given to trigger the download.

Testing and Monitoring

Testing the deployed agent involves subscribing to a device-specific MQTT topic and publishing a query. The blog post includes an example query to check the status of the conveyor belt and demonstrates the expected response format, including final and sub-agent responses. Monitoring the component’s operation is achieved by checking the component logs located at /greengrass/v2/logs/com.strands.agent.greengrass.log.

Conclusion

This post demonstrated how to deploy an SLM locally with AWS IoT Greengrass, showcasing the potential of edge AI in manufacturing and other sectors. The integration of SLMs through Strands Agents on constrained hardware allows for real-time decision-making and improved efficiency. The post concludes by envisioning a hybrid cloud-edge AI system where edge agents handle real-time processing and cloud agents manage complex reasoning, offering a scalable and adaptable solution for intelligent automation.

Source: AWS - Amazon Web Services