Amazon Alexa Voice Controlled Drone

In this project, I demonstrate developing a custom Alexa skill to fly and control a DJI Tello drone.

This project explores the key IoT platform aspects like device registry, device shadows (aka device twins) provided by AWS IoT service to enable communication between Raspberry Pi Zero, DJI Tello drone and a custom Amazon Alexa skill giving the ability to control the drone via voice commands.

In case you missed my first project on controlling DJI Tello drone with Xbox Controller, here is the link to it.

Here is a short video where I am flying the drone using Alexa voice commands:

Alexa Tello
If you are new to AWS and Raspberry Pi, you may feel a lot is going around to make this work but I promise it’s easy once you follow through the steps.

This project will also enable you to implement other ideas on similar lines or different since the blocks/services/concepts used here are very much needed for developing any IoT application.

Tech Stack

Following is the tech stack used for developing this project:

At high level, following are the sequence of events which take place during this interaction:

  1. User invokes the Alexa skill (in our case drone pilot) and issues a voice command
  2. Alexa skill validates this command with the available set of intents associated to the skill
  3. Alexa then sends the identified intent to the configured AWS Lambda function endpoint
  4. The lambda function receives incoming command,
    • Queries the device shadow service to check if the drone is online
    • Creates the command message and sends it to the AWS IoT device via MQTT channel
    • Responds to the Alexa command with a success/failure message
  5. A Raspberry Pi zero (connected to the DJI Tello via WIFI)
    • Subscribes to the AWS IoT MQTT channel for new messages.
    • On regular intervals, keeps on reporting the drone telemetry like speed, battery status, wifi strength to the AWS IoT device shadow
  6. Upon receiving a message, the the Raspberry Pi interprets the MQTT message and issues a corresponding DJI Tello specific command

image

Prerequisites

Hardware

You can follow the steps mentioned in the hardware section of my previous project to also prepare for the upcoming projects.

Alternatively, to keep it very simple here just connect the Micro USB to USB Type A female adapter + Wifi dongle to the Raspberry Pi (Zero).

Software

Provision AWS IoT Device

Let’s start by setting up a device in AWS IoT to enable us communication with Raspberry Pi.

Creating Policy

Creating a Thing

At this point, we have all the stuff ready to communicate with the Thing.

Creating AWS Lambda Function

Next step is to create an AWS Lambda function which will be invoked by the Alexa skill. The message passed by the Alexa invocation to the Lambda function will be validated against a list of allowed actions and further sent to the Thing we created in the above step.

Every message passed to the Lambda function represents a type of action the user wants to execute. Further every action has a designated MQTT topic defined in the policy attached to the Thing.

Creating Lambda Function

Packaging Lambda Function Code

thing_name = “"

- Save changes and close the file
- Open command line and type
`pip3 install AWSIoTPythonSDK -t .` to download AWSIoTPythonSDK inside the `lambda_function` directory
- Create a zip package with only the **contents** of the `lambda_function` directory

    `zip -9r lambda.zip AWSIoTPythonSDK* certs/* iot_client.py lambda_function.py alexa_response_builder.py`
- At this point you should have a zip file `lambda.zip` ready to be uploaded to AWS Lambda function

#### Upload Lambda Function Package
- Back on the Lambda function console, from the Designer section, click the lamda function icon
- Expand the `Code entry type` dropdown and select `Upload a .zip file`
- Click `Upload` button and browse and select the `lambda.zip` file
- Click `Save` button on the top right corner to save changes

You should now be able to see your code in the online code editor interface of AWS Lambda.

### Creating Alexa Skill
Now we are all set to create an Alexa skill which will interact with user to receive commands and fly the drone.

- Sign into [Alexa Skills Kit Developer Console](https://developer.amazon.com/alexa/console/ask) with your credential created in prerequisite section
- Click `Create Skill` button
- Give your skill a name e.g. `Tello Voice Control`
- Select a default language. Your skill will only appear if the user has this language selected
- Select the model as `Custom` and hosting method as `Provision your own`
- Click `Create skill` button on top right of the screen
- Select the skill template as `Start from scratch` and then click `Choose`
- Click `Invocation` from the left hand navigation
- Specify a Skill Invocation Name e.g. `drone pilot`
- Click `Save Model` button at the top of the page
- Click `JSON Editor` on the left hand navigation
- Copy and paste the contents of the `skill.json` file you downloaded as part of this repository into the editor surface
- Click `Save Model` button at the top of the page
- Feel free to go through the list of intents from the left hand navigation. Every intent represents a command the user can invoke
- Click `Endpoint` from the left hand navigation
- Select `AWS Lambda ARN`
- In the Default Region textbox paste the `ARN` of the Lambda function you noted in the previous section
- Click `Save Endpoints` button at the top of the page
- Click `Invocation` from the left hand navigation
- Click `Build Model` button at the top of the page and wait for the skill build process to complete

#### Test Alexa Skill
Now is the time to put your skill to test!!! 

There are multiple ways to test your skill. The easiest one is by using the Alexa simulator provided in the  Alexa Skills Developer Console
- On the top navigation bar, click `Test`
- Select `Skill testing is enabled in:` as `Development`
- Press the microphone button an speak _`Alexa, open drone pilot`_
- You should receive a voice feedback as _`To start, you should say: Alexa, ask drone pilot to take off.`_
- You can try other commands e.g. _`Alexa, ask drone pilot to take off`_ and alexa should respond back with the command acknowledgement
<img src="https://erviveksoni.github.io/alexa-controlled-drone/images/skill_card_image.png" alt="Alexa Skill Card" width="400" height="274" border="10" />
<br/>

#### Enabling Alexa Skill

- Sign in to https://alexa.amazon.com/ with your Alexa developer account credentials
- Navigate to `Skills` section in the left hand navigation
- Click `Your Skills` on top left of the screen
- Click `Dev Skills` on top navigation
- Click your skill name e.g. `Tello Voice Control`
- Click `ENABLE` button

This will enable your skill on all devices, iOS and Android apps connected to your Alexa developer account.

Congratulations!! You have successfully created an alexa skill to fly your drone!!

Now the last step is to setup Raspberry Pi and make it to talk to AWS IoT thing to receive commands.

### Setting up Raspberry Pi

#### Setting up Raspbian Buster Lite
We will setup Raspberry Pi in headless mode to get the optimal usage of RAM and CPU. There are many good posts on how to setup Raspbian Buster Lite on the Raspberry Pi Zero in [Headless Mode](https://desertbot.io/blog/setup-pi-zero-w-headless-wifi/) 

At this point in time, we should be able to SSH into out Pi using the Wifi onboard. Also the Pi will be most likely have access to the internet (dependeing on your WIFI network settings).

#### Connecting Raspberry Pi to Tello

When you turn on Tello, it configures itself as an AP allowing the clients to connect and control to it. Once a client is connected to Tello, it looses internet connectivity. 
To avoid this we'll configure the Raspberry Pi with dual WIFI interfaces. 

The Raspberry Pi onboard WIFI connects to the internet (via my home network) and the WIFI Adapter connects to Tello's WIFI.

Here are the steps:
- Ensure the WIFI dongle is connected to the Raspberry Pi Zero micro usb port
- Power on Raspberry Pi
- SSH into Raspberry Pi Zero
- Type `lsusb`. Ensure you see the WIFI USB adapter listed on the console output
- Type `sudo nano /etc/network/interfaces` to edit the network interfaces file
- Add the text below towards the end of the file. 
Replace the `TELLO_NETWORK_NAME` with the WIFI AP name of Tello followed by its password.

```c
auto lo

iface lo inet loopback
iface eth0 inet dhcp

allow-hotplug wlan0
iface wlan0 inet manual
wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf

allow-hotplug wlan1
iface wlan1 inet dhcp
wpa-ssid "<TELLO_NETWORK_NAME>"
wpa-psk "<PASSWORD>"

iface default inet dhcp
```
- Save your changes to the interfaces file
- Shutdown Raspberry Pi `sudo shutdown now`
- Turn on Tello
- Power on Raspberry Pi and SSH into it
- Type `ifconfig` to list the Raspberry Pi network interfaces
- You should see 2 interfaces `wlan0` and `wlan1` connected to their network respectively
- __In case you don't see an IP address acquired for `wlan1`__, then reset the `wlan1` interface using the command
 `sudo dhclient -v wlan1`

#### Installing Required Packages on Raspberry Pi
SSH into Raspberry Pi and follow the steps below.
##### Installing Python

- `sudo apt-get install python3-dev`
- `sudo apt install python3-pip`

##### Installing Required Packages
- `pip3 install AWSIoTPythonSDK`
- `pip3 install cpython`

##### Install TelloPy package
- `git clone https://github.com/hanyazou/TelloPy` 
- `cd TelloPy`
- `python3 setup.py bdist_wheel`
- `pip3 install dist/tellopy-*.dev*.whl --upgrade`

### Setting up the Source Code
- Clone this Repository on Raspberry Pi
  `git clone https://github.com/erviveksoni/alexa-controlled-drone`
- `cd` into the `alexa-controlled-drone/pi-alexa-code` directory
- Copy the [certs folder](#software) which has all the certificates from your development machine into `pi-alexa-code`
- Open `start.py` file in your preferred text editor 
- Update the config section at the top of this file with the cert names and Rest API Endpoint details you noted earlier 
Also, replace the `<THING_NAME>` with the Thing you created in [above section](#creating-a-thing)

````python
config = { 
         'host': '<REST API Endpoint>',
         'rootCAName': '<Root certificate file name>',
         'certificateName': '<Certificate file name>',
         'privateKeyName' : '<Private key file name>',
         'clientId': 'drone_alexa_client',
         'port' : 8883
}

thing_name = "<THING_NAME>"

Running the Application

Now its time to run the application!

Available Alexa Commands

Alexa, open drone pilot.
Alexa, ask drone pilot connection status.
Alexa, ask drone pilot status of battery. Other possible values (wifi/battery/camera)
Alexa, ask drone pilot to take off.
Alexa, ask drone pilot to go left. Other possible values (up/down/back/forward/left/right)
Alexa, ask drone pilot to rotate left. Other possible values (left/right)
Alexa, ask drone pilot to flip.
Alexa, ask drone pilot to land.