Adventure in the physical Web with microcontrollers (ESP32, Orange Pi Zero Plus 2 H5, TI CC2640R2) and a vocal chatbot in the form of a Progressive Web App ….

Image for post

I start with a new design once again :

Image for post

and I start by configuring my ESP32 card:

Image for post
Image for post
Image for post
Image for post

using the Mongoose OS framework available under Apache License Version 2.0. It supports low power connected microcontrollers such as: ESP32, ESP8266, TI CC3200, TI CC3220, STM32 …

Image for post
Image for post

I’m based on an existing github repository from Cesanta behind the framework Mongoose OS

Image for post
Image for post

and set the YAML mos.yml file :

Image for post
Image for post

As we can see it will load the sources of my chatbot dedicated to FranceConnect and expose it via an embedded web server. Build and flash the ESP32 card via the command line :

Image for post
Image for post
Image for post
Image for post

The card is set to connect to the local WiFi network :

The microcontroller has its address and exposes the voice chatbot via its embedded web server:

Image for post
Image for post

and I can check locally that the vocal chatbot responds well :

Image for post
Image for post

This vocal chatbot is in the form of a Progressive Web App linked to external cognitive APIs:

Image for post
Image for post
Image for post
Image for post

Reminder on the old articles on this subject :

and in autonomous mode, the ESP32 card remains accessible and controllable via a dashboard offered by Mongoose OS :

Image for post
Image for post

So I see the device online:

Image for post
Image for post
Image for post
Image for post

with information and particularly some logs …

Image for post
Image for post

that I can get in JSON via the REST API of the dashboard :

Image for post
Image for post

So, I will create on the Orange Pi Zero Plus 2 H5 card which is in ARMv8 64 Bits (unlike its little sisters Orange Pi Zero in ARMv7 32 Bits), a reverse proxy service in HTTPS to the embedded web server of the ESP32 card … Reminder of the technical features of this card :

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Again with the use of Armbian (Debian 9 Stretch version).

Image for post
Image for post

I create locally self-signed certificates that I will couple to a program simply named Reverse Proxy via Node.js :

Image for post
Image for post

I create this self-signed certificate using a Wildcard DNS address to simplify its recognition on a web browser :

Image for post
Image for post

and I run the command for the reverse proxy :

Image for post
Image for post
Image for post
Image for post

through a configuration file :

Image for post
Image for post

and I can see that locally with this reverse proxy in HTTPS, the chatbot responds (using an SSL certificate is important because the chatbot is actually in the form of a Progressive Web App or PWA) :

Image for post
Image for post

It finally remains for me to use the card TI CC2640R2 specialized in bluetooth to launch me in this test of the Physical Web. In autumn 2014, Google announced the creation of the Physical web, an open source project allowing all connected objects to communicate with a smartphone :

Image for post
Image for post

This microcontroller will actually be used as a Beacon broadcasting Eddystone-URL frames (URL address frames). To simplify the work of developers, Google has proposed two APIs associated with Eddystone. The Nearby API is designed to facilitate search by applications of nearby tags. The Proximity Beacon API allows developers to associate semantic location information and data stored in the cloud with tags :

Image for post
Image for post

To configure this device, I use Texas Instrument’s online development tools:

Image for post
Image for post

through its local agent who makes the link between the device and my web browser :

Image for post
Image for post
Image for post
Image for post

The Beacon sources are then loaded into the online IDE :

Image for post
Image for post

I create upstream the URL shortcut of the reverse proxy issued by the Orange Pi Zero Plus 2 card :

Image for post
Image for post

and I modify the sources to include it :

Image for post
Image for post

I only have to launch the “Build and Flash” to load this app into the microcontroller :

Image for post
Image for post

I then see via an application dedicated to Beacon scanning that my URL is well propagated locally :

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

This induces a notification on the smartphone inviting to open this URL which actually hides a Progressive Web App :

Image for post
Image for post

and a link is generated in the smartphone home screen :

Image for post
Image for post
Image for post
Image for post

If I don’t want to use a self-signed certificate to offer this voice chatbol in PWA outside, it is possible to use for example the localtunnel service which will automatically provide me with an endpoint in HTTPS :

Image for post
Image for post

It is installed on the Orange Pi Zero Plus 2 card with npm. I leave with a reverse proxy on the local HTTP port 8080 :

Image for post
Image for post

and I launch localtunnel which provides me the endpoint in HTTPS :

Image for post
Image for post

The chatbot running locally on the ESP32 board becomes externally accessible:

Image for post
Image for post

and with the Lighthouse extension, I can get the test scores for this PWA that are reasonable with this small WiFi connection:

Image for post
Image for post
Image for post
Image for post

Even if I had a better score before :

Image for post
Image for post

With the Physical Web, browsers move into the era of connected objects while freeing themselves from installation via online stores …

Image for post
Image for post

To be continued !

Above the clouds, the sky is always blue ...

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store