Adventure in the physical Web with microcontrollers (ESP32, Orange Pi Zero Plus 2 H5, TI CC2640R2) and a vocal chatbot in the form of a Progressive Web App ….

I start with a new design once again :

and I start by configuring my ESP32 card:

using the Mongoose OS framework available under Apache License Version 2.0. It supports low power connected microcontrollers such as: ESP32, ESP8266, TI CC3200, TI CC3220, STM32 …

I’m based on an existing github repository from Cesanta behind the framework Mongoose OS

and set the YAML mos.yml file :

As we can see it will load the sources of my chatbot dedicated to FranceConnect and expose it via an embedded web server. Build and flash the ESP32 card via the command line :

The card is set to connect to the local WiFi network :

The microcontroller has its address and exposes the voice chatbot via its embedded web server:

and I can check locally that the vocal chatbot responds well :

This vocal chatbot is in the form of a Progressive Web App linked to external cognitive APIs:

Reminder on the old articles on this subject :

Chatbot vocal de test utilisant Web Speech API et sous la forme d’une Progressive Web App …

Chatbot FranceConnect avec Azure Bot Services et Bing Speech API dans Kubernetes et DCOS …

and in autonomous mode, the ESP32 card remains accessible and controllable via a dashboard offered by Mongoose OS :

So I see the device online:

with information and particularly some logs …

that I can get in JSON via the REST API of the dashboard :

So, I will create on the Orange Pi Zero Plus 2 H5 card which is in ARMv8 64 Bits (unlike its little sisters Orange Pi Zero in ARMv7 32 Bits), a reverse proxy service in HTTPS to the embedded web server of the ESP32 card … Reminder of the technical features of this card :

Again with the use of Armbian (Debian 9 Stretch version).

I create locally self-signed certificates that I will couple to a program simply named Reverse Proxy via Node.js :

I create this self-signed certificate using a Wildcard DNS address to simplify its recognition on a web browser :

and I run the command for the reverse proxy :

through a configuration file :

and I can see that locally with this reverse proxy in HTTPS, the chatbot responds (using an SSL certificate is important because the chatbot is actually in the form of a Progressive Web App or PWA) :

It finally remains for me to use the card TI CC2640R2 specialized in bluetooth to launch me in this test of the Physical Web. In autumn 2014, Google announced the creation of the Physical web, an open source project allowing all connected objects to communicate with a smartphone :

This microcontroller will actually be used as a Beacon broadcasting Eddystone-URL frames (URL address frames). To simplify the work of developers, Google has proposed two APIs associated with Eddystone. The Nearby API is designed to facilitate search by applications of nearby tags. The Proximity Beacon API allows developers to associate semantic location information and data stored in the cloud with tags :

To configure this device, I use Texas Instrument’s online development tools:

through its local agent who makes the link between the device and my web browser :

The Beacon sources are then loaded into the online IDE :

I create upstream the URL shortcut of the reverse proxy issued by the Orange Pi Zero Plus 2 card :

and I modify the sources to include it :

I only have to launch the “Build and Flash” to load this app into the microcontroller :

I then see via an application dedicated to Beacon scanning that my URL is well propagated locally :

This induces a notification on the smartphone inviting to open this URL which actually hides a Progressive Web App :

and a link is generated in the smartphone home screen :

If I don’t want to use a self-signed certificate to offer this voice chatbol in PWA outside, it is possible to use for example the localtunnel service which will automatically provide me with an endpoint in HTTPS :

localtunnel

It is installed on the Orange Pi Zero Plus 2 card with npm. I leave with a reverse proxy on the local HTTP port 8080 :

and I launch localtunnel which provides me the endpoint in HTTPS :

The chatbot running locally on the ESP32 board becomes externally accessible:

and with the Lighthouse extension, I can get the test scores for this PWA that are reasonable with this small WiFi connection:

Even if I had a better score before :

With the Physical Web, browsers move into the era of connected objects while freeing themselves from installation via online stores …

To be continued !