A Small IoT Project -- Smart Refridgerator Assistant

Posted on Edited on In

Introduction

This small project is the final project for my CSS532 of UW CSS. Health is a significant topic as we always talk about, the foods we eat every day have very important effects on our body health. The refridgerator is the place where most of families store the foods, however, life is so busy that we also forget what we have put into our food store and what time we put them in. Normally, we have wasted the freshness of the good food even have some food gone bad and pollute the whole store. So, to address this, I want to create a smart refridgerator assistant who can take care the problems.

The idea is about recording the foods in your refridgerator and remind you of what we have and when we put them in. To implement a small prototype to make the requirements come true, I need to combine the techniques of IoT, Cloud computing and Machine Learning. With the help of this smart refridgerator assistant, we can avoid forgetting foods in our refridgerators and keep track on the freshness of them. The whole picture looks like that the assistant is going to monitor the refridgerator and send message to IoT hub, the cloud will store the food status and send notification to our mobile phone app and let us know the status of our refridgerator, so I can know what we have in our food store and decide the what to eat next.

Project Presentation

Technique Background

Devices

This project uses two devices, one is Raspberry Pi served as monitor device, the other is an Android mobile phone served as an end device. Two devices are not connected directly, AWS services act as the communication media. The lambda function will control the message flow, and all the messages are over MQTT protocal.

Raspberry Pi

The Raspberry Pi is a series of small single-board computers developed in the United Kingdom by the Raspberry Pi Foundation to promote teaching of basic computer science in schools and in developing countries. The one I use in this project is Raspberry Pi 3 Model B+. The Raspberry Pi is also equiped with a video camera for image capturing. 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Raspberry Pi 3 Model B+

CPU:     Quad core 64-bit processor clocked at 1.4GHz
Memory:  1GB LPDDR2 SRAM
Network: Dual-band 2.4GHz and 5GHz wireless LAN
         Bluetooth 4.2 / BLE
         Higher speed ethernet up to 300Mbps
Power:   Power-over-Ethernet capability (via a separate PoE HAT)
++++++++++++++

The Arducam 5 Megapixels 1080p Sensor

- High-Definition video camera for Raspberry Pi Model A or B, B+, model 2, Raspberry Pi 3,3 B+
- 5MPixel sensor with Omnivision OV5647 sensor in a fixed-focus lens
- Integral IR filter
- Still picture resolution: 2592 x 1944
- Max video resolution: 1080p

Raspberry Pi
Raspberry Pi

The Raspberry Pi device provides all runtime environment for the monitor part of this project, the device is installed with Raspbian and Python environment. The camera driver is provided by the default system.

Android Device (emulator)

As for Android Device, I use the android emulator provided by Google Android Studio for its pure Android OS. The client application will run in the emulator.
Android Device

AWS

AWS provides dozens of services that support our needs. In this project, I mainly use the service of IoT Hub, Dynamodb, Lambda, Cognito and CloudWatch.

IOT Hub

Lambda

Dynamodb

Cognito

CloudWatch

Machine Learning

In this project, the Raspberry Pi runs the recognition code

Image Recognition

Deep Learning

Caffe

System Design

Implementation

Reference