The tensorflow image processing platform allows you to detect and recognize objects in a camera image using TensorFlow. The state of the entity is the number of objects detected, and recognized objects are listed in the summary attribute along with quantity. The matches attribute provides the confidence score for recognition and the bounding box of the object for each detection category.

Notes for Home Assistant Core Installations


This integration requires files to be downloaded, compiled on your computer, and added to the Home Assistant configuration directory. These steps can be performed by cloning this repository into your configuration directory. Alternatively, if you wish to perform the process manually, the process is as follows:

Create the following folder structure in your configuration directory.

  |- {config_dir}
    |- tensorflow/
      |- models/

Follow these steps (Linux) to compile the object detection library.

# Clone tensorflow/models
git clone https://github.com/tensorflow/models.git
# Compile Protobuf (apt-get install protobuf-compiler)
cd models/research
protoc object_detection/protos/*.proto --python_out=.
# Copy object_detection to {config_dir}
cp -r object_detection {config_dir}/tensorflow

Your final folder structure should look as follows

  |- {config_dir}
    |- tensorflow/
      |- models/
      |- object_detection/
        |- ...

Model Selection

Lastly, it is time to pick a model. It is recommended to start with one of the COCO models available in the Model Detection Zoo.

The trade-off between the different models is accuracy vs speed. Users with a decent CPU should start with one of the EfficientDet models. If you are running on an ARM device like a Raspberry Pi, start with the SSD MobileNet v2 320x320 model.

Whichever model you choose, download it and extract in to the tensorflow/models folder in your configuration directory.


To enable this platform in your installation, add the following to your configuration.yaml file:

# Example configuration.yaml entry
  - platform: tensorflow
      - entity_id: camera.local_file
      graph: /config/tensorflow/models/efficientdet_d0_coco17_tpu-32/

Configuration Variables

source map Required

The list of image sources.

entity_id string Required

A camera entity id to get picture from.

name string (Optional)

This parameter allows you to override the name of your image_processing entity.

file_out list (Optional)

A template for the integration to save processed images including bounding boxes. camera_entity is available as the entity_id string of the triggered source camera.

model map Required

Information about the TensorFlow model.

graph string Required

Full path to the base model directory.

labels string (Optional)

Full path to a *label_map.pbtext.



label_offset integer (Optional, default: 1)

Offset for mapping label ID to a name (only use for custom models)

model_dir string (Optional)

Full path to TensorFlow models directory.


/tensorflow inside configuration

area map (Optional)

Custom detection area. Only objects fully in this box will be reported. Top of image is 0, bottom is 1. Same left to right.

top float (Optional, default: 0)

Top line defined as % from top of image.

left float (Optional, default: 0)

Left line defined as % from left of image.

bottom float (Optional, default: 1)

Bottom line defined as % from top of image.

right float (Optional, default: 1)

Right line defined as % from left of image.

categories list (Optional)

List of categories to include in object detection. Can be seen in the file provided to labels.

categories can also be defined as dictionary providing an area for each category as seen in the advanced configuration below:

# Example advanced configuration.yaml entry
  - platform: tensorflow
      - entity_id: camera.driveway
      - entity_id: camera.backyard
      - "/tmp/{{ camera_entity.split('.')[1] }}_latest.jpg"
      - "/tmp/{{ camera_entity.split('.')[1] }}_{{ now().strftime('%Y%m%d_%H%M%S') }}.jpg"
      graph: /config/tensorflow/models/efficientdet_d0_coco17_tpu-32/
        - category: person
            # Exclude top 10% of image
            top: 0.1
            # Exclude right 15% of image
            right: 0.85
        - car
        - truck

Optimizing resources

Image processing components process the image from a camera at a fixed period given by the scan_interval. This leads to excessive processing if the image on the camera hasn’t changed, as the default scan_interval is 10 seconds. You can override this by adding to your configuration scan_interval: 10000 (setting the interval to 10,000 seconds), and then call the image_processing.scan service when you actually want to perform processing.

# Example advanced configuration.yaml entry
  - platform: tensorflow
    scan_interval: 10000
      - entity_id: camera.driveway
      - entity_id: camera.backyard
# Example advanced automations.yaml entry
- alias: "TensorFlow scanning"
     - platform: state
         - binary_sensor.driveway
    - service: image_processing.scan
        entity_id: camera.driveway