silentfox
Posts: 5
Joined: Sat Aug 13, 2022 3:47 pm
Location: Talala, OK

weather_station_BYO not auto uploading to WU

Sat Aug 13, 2022 4:01 pm

Greetings fellow weather station builders,

So, I used the BYO guide on the raspberry pi foundations website, and was able to adapt the code to upload to Weather Underground. When I run the code in Idle, it works perfectly fine, and uploads to WU every 5 minutes. Now, I have tried to run my weather_station_BYU.py script in both rc.local, and as crontab, and running "ps aux | grep weather_station_BYO.py" shows that my script is running in both instances. BUT, it is NOT updating WU as the code should. I am not really sure what is going on with the script. Like I said, it runs perfectly in Idle every time. I have tried a fair few different solutions to starting both the rc.local and crontab, and it does show as running. I would be very grateful for any assistance.

I have attached my code for anyone with the time to look at it and give me some pointers. It does run perfectly fine in Idle, and updates WU, just not when I set it to run automatically when the Pi is started. The Pi is a rPi 3B+.

Thanks

from gpiozero import Button
import time
import math
import bme280_sensor
import wind_direction_byo
import statistics
import ds18b20_therm
import database
import requests

# ------- WU URL and credentials ---------

WUurl = "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php?"
WUid = "XXXXXXXX" # add your WS ID
WUpwd = "XXXXXXXX" # add your password
WUcreds = "ID=" + WUid + "&PASSWORD="+ WUpwd

wind_count = 0 # Counts how many half-rotations
radius_cm = 9.0 # Radius of the Anemometer
wind_interval = 5 # How often (seconds) to sample speed
interval = 300 # Measurements recorded every 5 minutes
CM_IN_A_KM = 100000.0
SECS_IN_AN_HOUR = 3600
ADJUSTMENT = 1.18
BUCKET_SIZE = 0.2794
rain_count = 0
gust = 0
store_speeds = []
store_directions = []

# ------- Convert units from Metric to Standard -------

def pa_to_inches(pressure_in_pa):
pressure_in_inches_of_m = pressure_in_pa * 0.02953
return pressure_in_inches_of_m

def mm_to_inches(rainfall_in_mm):
rainfall_in_inches = rainfall_in_mm * 0.0393701
return rainfall_in_inches

def khm_to_mph(speed_in_kmh):
speed_in_mph = speed_in_kmh * 0.621371
return speed_in_mph

def degc_to_degf(temperature_in_c):
temperature_in_f = (temperature_in_c * (9/5.0)) + 32
return temperature_in_f

#Every half-rotation, add 1 to count
def spin():
global wind_count
wind_count = wind_count + 1
#print( wind_count )

def calculate_speed(time_sec):
global wind_count
global gust
circumference_cm = (2 * math.pi) * radius_cm
rotations = wind_count / 2.0

# Calculate distance travelled by a cup in km
dist_km = (circumference_cm * rotations) / CM_IN_A_KM

#Speed = Distance / Time
km_per_sec = dist_km / time_sec
km_per_hour = km_per_sec * SECS_IN_AN_HOUR

# Calculate Speed
final_speed = km_per_hour * ADJUSTMENT

return final_speed

def bucket_tipped():
global rain_count
rain_count = rain_count + 1
#print (rain_count * BUCKET_SIZE)

def reset_rainfall():
global rain_count
rain_count = 0

def reset_wind():
global wind_count
wind_count = 0

def reset_gust():
global gust
gust = 0

wind_speed_sensor = Button(5)
wind_speed_sensor.when_activated = spin
temp_probe = ds18b20_therm.DS18B20()

db = database.weather_database()

while True:
start_time = time.time()
while time.time() - start_time <= interval:
wind_start_time = time.time()
reset_wind()
#time.sleep(wind_interval)
while time.time() - wind_start_time <= wind_interval:
store_directions.append(wind_direction_byo.get_value())

final_speed = calculate_speed(wind_interval) # Add this speed to the list
store_speeds.append(final_speed)
wind_average = wind_direction_byo.get_average(store_directions)
wind_gust = max(store_speeds)
wind_speed = statistics.mean(store_speeds)
rainfall = rain_count * BUCKET_SIZE
reset_rainfall()
store_speeds = []
print(store_directions)
store_directions = []
ground_temp = temp_probe.read_temp()
humidity, pressure, ambient_temp = bme280_sensor.read_all()
ambient_temp_f = float("{0:.2f}".format(degc_to_degf(ambient_temp)))
ground_temp_f = float("{0:.2f}".format(degc_to_degf(ground_temp)))
ambient_temp = float("{0:.2f}".format(ambient_temp))
ground_temp = float("{0:.2f}".format(ground_temp))
humidity = float("{0:.2f}".format(humidity))
pressure_in = float("{0:.2f}".format(pa_to_inches(pressure)))
pressure = float("{0:.2f}".format(pressure))
wind_speed_mph = float("{0:.2f}".format(khm_to_mph(wind_speed)))
wind_speed = float("{0:.2f}".format(wind_speed))
wind_gust_mph = float("{0:.2f}".format(khm_to_mph(wind_gust)))
wind_gust = float("{0:.2f}".format(wind_gust))
wind_average = float("{0:.2f}".format(wind_average))
rainfall_in = float("{0:.2f}".format(mm_to_inches(rainfall)))
rainfall = float("{0:.2f}".format(rainfall))
print(ambient_temp, ground_temp, humidity, pressure, wind_speed, wind_gust, wind_average, rainfall)
print(ambient_temp_f, ground_temp_f, humidity, pressure_in, wind_speed_mph, wind_gust_mph, wind_average, rainfall_in)
db.insert(ambient_temp, ground_temp, 0, pressure, humidity, wind_average, wind_speed, wind_gust, rainfall)
# Weatherunderground
print("uploading to Weatherunderground")
f_date = "&dateutc=now"
f_humid = "&humidity=" + str(humidity)
f_wspeed = "&windspeedmph=" + str(wind_speed_mph)
f_gust = "&windgustmph=" + str(wind_gust_mph)
f_airtemp = "&tempf=" + str(ambient_temp_f) # degrees F
f_rain = "&rainin=" + str(rainfall_in)
f_press = "&baromin=" + str(pressure_in) # inches
f_groundtemp = "&soiltempf=" + str(ground_temp_f) # degrees F
f_winddir = "&winddir=" + str(wind_average)
f_action = "&action=updateraw"
r= requests.get(WUurl+WUcreds+f_date+f_humid+f_wspeed+f_winddir+f_gust+f_airtemp+f_rain+f_press+f_groundtemp+f_action)
print("Received " + str(r.status_code) + " " + str(r.text) + " from WU")

User avatar
neilgl
Posts: 4955
Joined: Sun Jan 26, 2014 8:36 pm
Location: Near The National Museum of Computing

Re: weather_station_BYO not auto uploading to WU

Sun Aug 14, 2022 9:17 am

When you run it from pi’s crontab (not root crontab), if you direct the output to a log file, what do you see?
Probably something that is on the correct path when run in Idle is not when run from cron.
See the excellent guide https://forums.raspberrypi.com/viewtopic.php?t=314455

silentfox
Posts: 5
Joined: Sat Aug 13, 2022 3:47 pm
Location: Talala, OK

Re: weather_station_BYO not auto uploading to WU

Sun Aug 14, 2022 2:44 pm

So, to check to see if I had an error, in Crontab I had it send an error to a txt document. Showed I had database errors. I then looked at a few forum posts here to see what I might be missing. Sure enough I was missing the MySQLdb import.

Upon fixing that error, my error log shows nothing in it at all now, but it still is not updating weather underground for some reason. So, not really sure what is going on with this thing. I can manually run it, even in a terminal and it works fine, it just, for some reason, doesn't want to run automatically on boot.

User avatar
neilgl
Posts: 4955
Joined: Sun Jan 26, 2014 8:36 pm
Location: Near The National Museum of Computing

Re: weather_station_BYO not auto uploading to WU

Sun Aug 14, 2022 3:02 pm

The log should show something where you update WU ?
print("Received " + str(r.status_code) + " " + str(r.text) + " from WU")
What was you crontab entry?

silentfox
Posts: 5
Joined: Sat Aug 13, 2022 3:47 pm
Location: Talala, OK

Re: weather_station_BYO not auto uploading to WU

Sun Aug 14, 2022 6:18 pm

When I found the database error my crontab was this:

@reboot python3 /home/pi/weather-station/weather_station_BYO.py >>/home/pi/error.txt 2>&1

After I added the import MySQLdb I am not getting anything in the error log I have setup at all.

My current crontab entry is this:

@reboot python3 /home/pi/weather-station/weather_station_BYO.py

I have tried Sudo in front of python3, I have tried it in a Sudo crontab both with and without sudo in the script call, I set the rPi to wait for network before completing boot, I tried it in rc.local as "python3 /home/pi/weather-station/weather_station_BYO.py start" with and without sudo.

When I run the code manually, WU gives me a good reply that the data was transferred and it shows as online.

User avatar
neilgl
Posts: 4955
Joined: Sun Jan 26, 2014 8:36 pm
Location: Near The National Museum of Computing

Re: weather_station_BYO not auto uploading to WU

Sun Aug 14, 2022 6:45 pm

Add a -u to the command (-u means unbuffered output)
@reboot python3 -u /home/pi/weather-station/weather_station_BYO.py >>/home/pi/error.txt 2>&1

And add a few print statements to the code so you can see where it gets to.

silentfox
Posts: 5
Joined: Sat Aug 13, 2022 3:47 pm
Location: Talala, OK

Re: weather_station_BYO not auto uploading to WU

Sun Aug 14, 2022 8:10 pm

Ok, so after checking permissions of files, and fixing a few, I was finally able to get an output into my error document, the following is my error:
Past Import statements
Past WU credentials
Past global object creation
Past conversion definitions
Past global rainfall and wind definitions
Traceback (most recent call last):
File "/home/pi/weather-station/weather_station_BYO.py", line 105, in <module>
db = database.weather_database()
File "/home/pi/weather-station/database.py", line 114, in __init__
self.db = mysql_database()
File "/home/pi/weather-station/database.py", line 25, in __init__
self.connection = MySQLdb.connect(user=credentials["USERNAME"], password=credentials["PASSWORD"], database=credentials["DATABASE"])
File "/usr/local/lib/python3.7/dist-packages/MySQLdb/__init__.py", line 130, in Connect
return Connection(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/MySQLdb/connections.py", line 185, in __init__
super().__init__(*args, **kwargs2)
MySQLdb._exceptions.OperationalError: (2002, "Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)")
Exception ignored in: <function mysql_database.__del__ at 0x75a3b300>
Traceback (most recent call last):
File "/home/pi/weather-station/database.py", line 42, in __del__
self.connection.close()
AttributeError: 'mysql_database' object has no attribute 'connection'
So it appears I still have a database issue when it runs through cron.

User avatar
neilgl
Posts: 4955
Joined: Sun Jan 26, 2014 8:36 pm
Location: Near The National Museum of Computing

Re: weather_station_BYO not auto uploading to WU

Mon Aug 15, 2022 8:25 am

Probably because the database is not yet up and running when the script runs.
Add a delay to the @reboot or a sleep() in your script or edit the .service file to make a dependency on mariadb running.
My quick test to write a weather record used this:

Code: Select all

@reboot sleep 30 && python -u /home/pi/scripts/dbwrite.py >> /home/pi/scripts/log.txt 2>&1
That explains why it runs ok when you log in and run it - the database is up.

silentfox
Posts: 5
Joined: Sat Aug 13, 2022 3:47 pm
Location: Talala, OK

Re: weather_station_BYO not auto uploading to WU

Mon Aug 15, 2022 11:57 pm

That fixed my issue, thank you so much for your time and assistance!

User avatar
neilgl
Posts: 4955
Joined: Sun Jan 26, 2014 8:36 pm
Location: Near The National Museum of Computing

Re: weather_station_BYO not auto uploading to WU

Tue Aug 16, 2022 6:32 pm

Good one

Return to “Weather station”