Smart Fridge Part 2

In a previous article, I wrote about IoT, the motivation behind this project and more. In this one, I’ll elaborate on the technical side of this project.

Let’s divide the project to two parts:

1. Taking a photo
2. Uploading the photo to Google Drive

This is how we can capture the content. Credit: Roi Natan Zukerman

Say Cheese

Since we want to save costs, we would like to use only one camera for capturing the food. Thus, we need a wide field of view. We can achieve it by:

1. Using a fisheye lens
2. Increasing the distance between the camera and the food

Those are the reasons for me to buy an ultra-wide 170° lens and install it on the fridge’s door. This worked perfectly… in theory.

The software doesn’t know when the door is open with “the right angle” for capturing the food. Thus, it keeps taking photos indefinitely (every couple of milliseconds). When the door is closed, there is no light in the fridge, so we don’t want to save such a photo. How can we know without installing more hardware if the door is closed or not?

I wrote a code the checks the photo. If it’s too dark, it takes another photo without saving the dark one.

The camera I bought supports automatic exposure. It means that when the photo it too dark, it increases automatically the exposure time, so the photo will be brighter. When we open the door, there is a lot of light that’s being captured by the camera which makes the photo very bright.

I solved this issue in a similar approach to the previous, too dark photos, issue. If the photo is too bright, it takes another one without saving the bright one.

Photo by NastyaSensei on Pexels.com

Uploading

There is a Python module for Google Drive which is great. It needs the credentials for creating a connection. Unfortunetally, uploading those photos might take a while since the uploading speed is inconsistent. Thus, taking a photo and immediately upload it using the same thread is problematic since we might miss “right angle” photos.

Ergo, the progam manages an uploading queue for pending photos. The SmartFridge class which runs on the main thread, initializes the FileUploader class which starts another thread. After taking a clear photo, the SmartFridge object tell to the FileUploader to upload a file. It pushes the file path to a queue. Then, the other thread gets an item from the queue and uploads it.

file_path = self.__camera.capture()
	if file_path:
		if photoshop.is_photo_clear(file_path):
			photoshop.rotate_photo(file_path)
			self.__fileUploader.upload_file(file_path)
		else:
			os.remove(file_path)

Every couple of miliseconds the program executes this code. It captures a photo, receives the file path, checks if the photo is clear (not too bright/dark), rotates it if needed and tells the file uploader to upload the photo. If the photo is unclear, it removes the file.

Limited storage volume

The (free) Google Drive storage volume is finate. Hence, we need to remove old files sometimes.

Before uploading new files, we remove the old one if there are too many.

Photo by Pixabay on Pexels.com

Improvements

There are more features I would like to implement for this project such as:

1. Supporting more than one camera
2. Adding LEDs to the fridge for better lighting
3. Identifing the “right angle” for shooting using software only
4. Dropping blurry photos
5. interpretering the fridge content by machine learning

Fork it

If you want to make your fridge smarter, you can fork the repository easily. Code imporvements are welcome as always.

This is the end result

---

If you have any questions feel free to comment or contact me directly.

Thank you for reading this. Please share and visit soon again,

Orian Zinger.