AWS DeepLens is a deep learning enabled developer toolkit with a video camera. It enables you to develop machine learning skills using hands-on computer vision tutorials, pre-built models and allows you to extend them.

This blog post explains how to extend the local functionality of DeepLens with cloud functionality using the AWS IoT Rule Engine and a Lambda function. The simple functionality that we describe here is to send an SMS notification to your phone number after you see a hot dog with your DeepLens device. We expect more advanced users to extend this functionality to include other AWS Cloud services such as

Here is a diagram that shows the flow of the data in the system from the object in front of the camera all the way to the mobile device in your pocket.

Create Your Lambda Function

First, you’ll create an AWS Lambda function that will run in the cloud and filter the messages that are coming from your DeepLens device for ones with high enough (>0.5) probability for a hot dog. During this process you will also create a rule in the AWS IoT rule engine to get messages from the Lambda function that you deployed to the device using AWS Greengrass.

• On the AWS Lambda console go to“Create Function.”
• Filter the Blueprints with “iot-button-email” and choose it as your blueprint template.
• Give your Lambda function a name. For example “Hotdog_Notifier.”
• Keep the “Create a new Role from template(s)” value in the Role field.
• Give your new role a name. For example, “Hotdog_Notifier_Role.”
• Add in the Policy Templates the policy for “SNS Publish policy.”
  
• In the “aws-iot” section switch to use “Custom IoT Rule”
  • Choose “Create a new rule.”
• Give it a name (for example, “search_hotdogs”) and a description.
• In the Rule query statement put the SELECT query: Select Hotdog from ‘/$aws/deeplens/KJHFD-DKJO87-LJLKD/inference’. This query can catch messages from the DeepLens device in the following JSON format:  { "Hotdog" : 0.5438 }
• Enable the Trigger in the checkbox that follows.
  
• We will modify the code of the Lambda function in the next step.
• Change the environment parameters from “email” to “phone_number”, and put your phone number as the Value. Please note that the phone number format should include the international country code (for example, for US +15555555555). You can read more on the international support for SMS in AWS SNS FAQ:  https://aws.amazon.com/sns/faqs/#sms-related-questions
• Choose the “Create Function” button.
  
• Switch to the “Configuration” of the Lambda function that you just created. You can find configuration tab on the left (configuration, triggers, and monitoring)
• In the Lambda function code we can remove all the helper functions that are needed for a usual SNS subscription, such as findExistingSubscription, createSubscription, createTopic.  Remove all the code until ‘use strict’. We will also modify the code to directly send the SMS:

  'use strict';
  
  /**
   * This is a sample Lambda function that sends an SMS Notification When your
   * Deep Lens device detects a Hot Dog
   * 
   * Follow these steps to complete the configuration of your function:
   *
   * Update the phone number environment variable with your phone number.
   */
  
  const AWS = require('aws-sdk');
  
  const phone_number = process.env.phone_number;
  const SNS = new AWS.SNS({ apiVersion: '2010-03-31' });
  
  exports.handler = (event, context, callback) => {
      console.log('Received event:', event);
  
      // publish message
      const params = {
          Message: `Your DeepLens device just identified a Hot Dog. Congratulations!`,
          PhoneNumber: phone_number
      };
      if (event.Hotdog > 0.5)
          SNS.publish(params, callback);
  };

• Choose “Save.”. You can test the Lambda function from this screen as well, but we will test it through the IoT rule engine now, to simulate the flow of messages from the DeepLens device.

Test Your Configuration

• In the IoT console select the “Test” option.
• Choose Publish to a Topic and Publish to the topic you defined in your Rule above the following message { "Hotdog": 0.68725 }.
  
• You should receive an SMS notification with the message: “Your DeepLens device just….” that you defined in your Lambda function.
• Time to show your DeepLens device some objects and a hot dog. Good Luck!
• If you don’t get messages when a hot dog is presented to the device, please go back to the diagram on top and verify that you aligned the values according to the colors, and that you provided the right phone number with the country code.

Conclusion

Amazon DeepLens is designed as an open platform for education and innovation, and we expect developers to come with many different ideas to use it to solve real life problems. From analyzing people walking around a store to automatically open the garage door to your car, or alerting you when tables in your restaurant are dirty or customers are asking for a service. This is only a small sample of systems you can build with AWS DeepLens using extensions similar to the simple one in this blog post.

Additional Reading

About the Author

Guy Ernest is a principal solutions architect in Amazon AI. He has the exciting opportunity to help shape and deliver on a strategy to build mind share and broad use of Amazon’s cloud computing platform for AI, machine learning and deep learning use cases. In his spare time, he enjoys spending time with his wife and family, gathering embarrassing stories, to share in talks about Amazon and the future of AI.