jsmwrench
Posts: 19
Joined: Sun Jan 09, 2022 5:26 pm

sending/recieving data from Pi to local web page

Sun Nov 20, 2022 4:20 pm

Hello!

I'm looking for instruction/advise/direction for a project I'm working on. I have written a local web page that will act as a control panel for a piece of equipment being built. The control panel/web page is hosted on a Raspberry Pi and set in kiosk mode and seems to work find for now. I'm looking for assistance in two areas.

:?: 1) How do I get data/commands from the Pi to the web page and/or web page to the Pi?

:?: 2) How do I send this data/commands from the Pi to my Arduinos that will be controlling the machine (stepper motors, lights, sensors, etc.)?

Any help/assistance will be appreciated.
Thank you,
Ron.

P.S. I know the basics of HTML and CSS and have learned quite a bit of Arduino programming, but I'm completely new to the Pi/Python world.

jayben
Posts: 465
Joined: Mon Aug 19, 2019 9:56 pm

Re: sending/recieving data from Pi to local web page

Sun Nov 20, 2022 4:44 pm

jsmwrench wrote:
Sun Nov 20, 2022 4:20 pm
1) How do I get data/commands from the Pi to the web page and/or web page to the Pi?
2) How do I send this data/commands from the Pi to my Arduinos that will be controlling the machine (stepper motors, lights, sensors, etc.)?
Take a look at my remote network analyser project here https://iosoft.blog/edla-part3 which has multiple networked units controlled from a browser, but to answer you questions directly:

1a: Pi to Web page: you can use server-side includes, AJAX, or on-demand page creation; see my explanation in https://iosoft.blog/pico-wireless-web-esp32. For machinery control, you could have a static HTML page, that is modified using Javascript from a JSON file, e.g. an entry in the file might indicate the machine is running, the Javascript application parses the file, and inserts the word 'running' at a suitable place in the page.

1b. Web page to Pi. Using Javascript handlers attached to the controls on your (static) Web page.

2. It depends what sort of interface you want between the Pi and Arduinos - the simplest is probably asynchronous serial, or use could use networking, maybe with TCP data & status links similar to those I used in the logic analyser.

Or you could just take the easy way out, and run a SCADA front-end, which is specifically designed for machine control, perhaps with the Arduino units talking Modbus over a serial or a network connection - there are a lot of possibilities.

User avatar
neilgl
Posts: 5317
Joined: Sun Jan 26, 2014 8:36 pm
Location: Near The National Museum of Computing

Re: sending/recieving data from Pi to local web page

Sun Nov 20, 2022 6:44 pm

Maybe start with the basics.
Have you written the web page in html (yes) and are you running a webserver (apache2 or nginx etc.) on the pi (no) ?
If you install and run apache2 on the pi, you can use php and python scripts with that to send commands to and get results from the pi.
Other people use python flask for the page and commands, or node-red or others.

jsmwrench
Posts: 19
Joined: Sun Jan 09, 2022 5:26 pm

Re: sending/recieving data from Pi to local web page

Thu Nov 24, 2022 8:25 pm

jayben and neilgl,

First of all, thank you both for taking the time to share your suggestions. Secondly, I think I'm in over my head, but I'll keep plugging along. Hopefully you, or someone as helpful as you will be around for my next round of questions.

jayben,
It sounds like you really know your way around programming microcontrollers, but I can only comprehend some of what you explained. I know what some of the terms you used and how to do some of what you wrote, but there is still a huge gap in my education to be able to follow.

neilgl,
I guess I'm a little lost on the "webserver" approach. I have the Pi set up in kiosk mode and can launch my web page directly into a Chromium browser upon starting the Pi. What would Apache2 do for the system? How do I get it to interact with my web page?

As an after thought, would either one of you be interested in assisting me in this project privately (off of the forum)? Maybe through email and/or Teams/Zoom? If so, please let me know.

Thank you again for your time.
Ron.

User avatar
neilgl
Posts: 5317
Joined: Sun Jan 26, 2014 8:36 pm
Location: Near The National Museum of Computing

Re: sending/recieving data from Pi to local web page

Thu Nov 24, 2022 11:05 pm

Apache2 on the pi (the server) will allow you to run other stuff on that server e.g. control equipment (via php, cgi etc.) connected to that server. It also “serves” up web pages, images, css, javascript etc.
Off the forum - not for me.

jsmwrench
Posts: 19
Joined: Sun Jan 09, 2022 5:26 pm

Re: sending/recieving data from Pi to local web page

Fri Nov 25, 2022 2:10 am

I appreciate the info, neilgl. I got apache2 installed today, but haven't found much yet on how to use it with an offline system. I'll browse around tomorrow and see if I can find a tutorial that will point me in the right direction. Thank you again for your time.

User avatar
neilgl
Posts: 5317
Joined: Sun Jan 26, 2014 8:36 pm
Location: Near The National Museum of Computing

Re: sending/recieving data from Pi to local web page

Fri Nov 25, 2022 10:39 am

To use it in an “offline” system (no internet connection wanted/needed) we can log in to the pi desktop and use chromium to open a web page that you have created in /var/www/html e.g. open a terminal, then

Code: Select all

cd /var/www/html
sudo nano mypage.html
save and exit. In chromium open localhost/mypage.html.

jsmwrench
Posts: 19
Joined: Sun Jan 09, 2022 5:26 pm

Re: sending/recieving data from Pi to local web page

Fri Nov 25, 2022 1:51 pm

neilgl wrote:
Fri Nov 25, 2022 10:39 am
To use it in an “offline” system (no internet connection wanted/needed) we can log in to the pi desktop and use chromium to open a web page that you have created in /var/www/html e.g. open a terminal, then

Code: Select all

cd /var/www/html
sudo nano mypage.html
save and exit. In chromium open localhost/mypage.html.
Thank you, again. Is there a way to "point" to my web page on an external drive like: /media/web_pages/mywebpage? Or does it have to be in the /var/www/html folder to work?

User avatar
neilgl
Posts: 5317
Joined: Sun Jan 26, 2014 8:36 pm
Location: Near The National Museum of Computing

Re: sending/recieving data from Pi to local web page

Fri Nov 25, 2022 2:35 pm

You can configure apache2 to use a different directory, and allow use “www-data” read/write permissions on that and files in it.
User www-data is the user Apache runs under (unless changed).
For starting out, use var/www/html. You can copy an existing file there (sudo cp) from your other directory.
Reason - start with default standard on everything, then it is easier for others to check/ replicate.

jsmwrench
Posts: 19
Joined: Sun Jan 09, 2022 5:26 pm

Re: sending/recieving data from Pi to local web page

Fri Nov 25, 2022 2:49 pm

That makes sense. Okay. Thank you again. I really appreciate the assistance.

pidd
Posts: 4153
Joined: Fri May 29, 2020 8:29 pm
Location: Wirral, UK

Re: sending/recieving data from Pi to local web page

Fri Nov 25, 2022 3:02 pm

neilgl wrote:
Fri Nov 25, 2022 2:35 pm
You can configure apache2 to use a different directory, and allow use “www-data” read/write permissions on that and files in it.
User www-data is the user Apache runs under (unless changed)
The files should only be read access by www-data except for any files you want apache to write to (which would be unusual/rare).

Generally ownership is best to be root to ensure security. I tend to use root ownership and www-data group with 640 permissions although there are other options.

jsmwrench
Posts: 19
Joined: Sun Jan 09, 2022 5:26 pm

Re: sending/recieving data from Pi to local web page

Sat Nov 26, 2022 3:44 pm

Hi all.

Again, thank you for all of the support you've given so far. I realize I've taken on a project larger than my experience, but I believe I can get through this if I just take one step at a time. To explain my project as simply as possible here is the goal:

-- There is going to be an equipment "Control Panel" on a screen used to control the equipment and give readouts to the user
-- Inputs from the user on the Control Panel will be sent to a Raspberry Pi (which is hosting the web page/Control Panel)
-- The Pi will then signals to one of two different Arduinos that will actually control the equipment
-- Simultaneously, the Arduinos will be sending sensor and mechanical information to the Pi
-- The Pi will then "post" those readings on the web page/Control Panel


This is what I've accomplished so far:

-- The Arduinos are programmed and operate the machinery perfectly.
-- I have used HTML/CSS to make a web page to act as a Control Panel
-- I've got a Raspberry Pi installed with an OS (set up in Kiosk mode), Apache2, and mySQL
-- The web page has been successfully loaded onto the Raspberry Pi and displays properly
-- Using JavaScript and HTML/CSS, I have all of the buttons working as far as indicating whether they have been selected/pushed


So I guess my next step and question is: How do I get the signal that the button has been pushed to Raspberry Pi? I suppose I'll also have to know: How do I put this signal or command into a variable to be stored and retrieved on the Pi?

Ron.

jayben
Posts: 465
Joined: Mon Aug 19, 2019 9:56 pm

Re: sending/recieving data from Pi to local web page

Sat Nov 26, 2022 7:58 pm

When the user clicks a button on a Web page, it triggers an event on the browser, which can basically do one of two things:

1. Load a new Web page. This isn't normally desirable, since you just want to send some data out, without a dramatic change to what the user sees.

2. Trigger a Javascript event. If your Web page has some Javascript code attached, the button will run a specific function on the browser.

Since Javascript is a complete language, you can theoretically do (almost) anything you like. The code is running on the browser, and you say you want to change a variable on the Web server, so the Javascript needs to initiate a connection to the server, and send it the new data.

One way to do that is to send an HTML 'post' to the server with the data; another way is to send a 'get' with some parameters, as I've previously described. These are standard techniques, and you can find numerous examples on the Web, under the general heading of AJAX.

However, a problem with this approach is that once you have pushed the new data to the server, you then need to trigger the other remote units to fetch that new data from the server, which is yet more complication, so I've suggested that you send the data directly from the browser to the remote units, rather than going via the Web server, as I've done in the distributed logic analyser project I mentioned.

At this moment you're probably thinking this is all too complicated, and you don't want to learn Javascript or Web protocols, but if you try to avoid them you could end up with something that is very clunky and inefficient. So either you find a wonder GUI package that does everything you want as standard (e.g. a SCADA application) or you need dip your toe into the complicated world of Web scripting.

jsmwrench
Posts: 19
Joined: Sun Jan 09, 2022 5:26 pm

Re: sending/recieving data from Pi to local web page

Sat Nov 26, 2022 8:13 pm

jayben,

This is great information! Thank you. And FYI, I really do want to learn JavaScript and the Web protocols. But you are correct, this seems very complicated....but I'm willing to try and learn it. My problem is (and I know this isn't your problem, just explaining) I've already taken on this project and have gotten this far with it. If I can just accomplish a couple of successful communications between the web page, Pi, and Arduinos, I think would be able to use that education to finish my part of this project.

So, if I'm understanding you correctly, I could just send the data directly to the Arduinos from the web page? I'm guessing then that I could also send my read-out data to the web page directly from the Arduinos? This would work great for my Power buttons, Reset buttons, and my stepper motor control Buttons. Would this use JavaScript too, or is there another protocol/method I need to look at?

I still think I'll need some communication between the page and the Pi, though. I'll have numerous stepper motor positions to record while using the equipment and I'll need to recall those positions each time the equipment is used. I know that will require database knowledge, but I'll tackle that after I get the "direct" data transfer problem learned.

Thank you again for your time.
Ron.

jayben
Posts: 465
Joined: Mon Aug 19, 2019 9:56 pm

Re: sending/recieving data from Pi to local web page

Sun Nov 27, 2022 10:23 am

Yes, you can access any of your networked devices from the Web page using Javascript; I won't reproduce all of the source code for my EDLA project here, but for example here is the function I use to send commands to a remote unit, and get its status.

Code: Select all

// Get status from unit, optionally send command
    function get_status(cmd=null) {
      http_request = new XMLHttpRequest();
      http_request.addEventListener("load", status_handler);
      http_request.addEventListener("error", status_fail);
      http_request.addEventListener("timeout", status_fail);
      var params = formParams(cmd), statusfile=remote_ip()+'/'+statusname;
      http_request.open( "GET", statusfile + "?" + encodeURI(params.join("&")));
      http_request.timeout = 2000;
      http_request.send();
    }  
This is overly complicated for your application, and I've omitted some helper functions, but the actions are basically:
  • Create an HTTP request
  • Set up event handlers for success or failure
  • Get parameters for the request from a form on the Web page (this means that you only need to declare the parameters once, in the page HTML, rather than managing the parameter list in Javascript)
  • Optionally add in a 'cmd' parameter to send a command to the remote
  • Get the URL for the unit we're addressing, by combining the filename with the IP address selected by the user
  • Create the GET request, with a timeout for completion. Create a parameter string with '&' delimiters. URL-encode the string, and place it after the GET, with a '?' prefix.
  • Send the request
After all this you should either get a response or failure event. The unit returns its data in JSON format, so it is easy to parse with Javascript

Code: Select all

// Decode status response
    function status_handler(e) {
      var remote_status = JSON.parse(event.target.responseText);
      var state = remote_status.state;
      ..and so on..
This parses the file, and copies a variable 'state' from it.
The same technique can be used to update your main Web server, if necessary.
Full explanation and source code at https://iosoft.blog/edla

jsmwrench
Posts: 19
Joined: Sun Jan 09, 2022 5:26 pm

Re: sending/recieving data from Pi to local web page

Sun Nov 27, 2022 4:36 pm

jayben,

Again, thank you! This is a lot of information that I'll have to process, understand, and learn. I'll work on learning as much as I can today, but with the work-week coming on, I probably won't accomplish anything until I have more time next weekend.

I appreciate all of the time you've spent sharing this info with me. I thought I had broken this down to the smallest "steps" to learn in order to complete this, but by the volume of information you provided that I have still yet to understand, I can see I've got to break this down into even smaller steps.

I'll be back in touch as soon as I think I've gotten somewhere with this project.

Ron.

User avatar
neilgl
Posts: 5317
Joined: Sun Jan 26, 2014 8:36 pm
Location: Near The National Museum of Computing

Re: sending/recieving data from Pi to local web page

Sun Nov 27, 2022 5:29 pm

If you are going the javascript route, it may be worth looking at mqtt (mqttws31.js), then you could do

Code: Select all

mqtt.send(message);
and have the networked devices listen for messages.

jsmwrench
Posts: 19
Joined: Sun Jan 09, 2022 5:26 pm

Re: sending/recieving data from Pi to local web page

Sun Nov 27, 2022 6:37 pm

Neilgl,

Thank you also! I guess I'll have to look into that as well. I appreciate your time.

Ron.

User avatar
neilgl
Posts: 5317
Joined: Sun Jan 26, 2014 8:36 pm
Location: Near The National Museum of Computing

Re: sending/recieving data from Pi to local web page

Sun Nov 27, 2022 6:45 pm

Amongst the advantages of mqtt are that there are many examples around, many public test servers (brokers) exist, you can look at all messages sent and received, you can run your own server on a pi, devices can have a “last will and testament” so you know if they have gone offline, etc.

Return to “General programming discussion”