Burst-upload in the morning

Topics: 5. Support
Jan 1, 2015 at 12:23 AM
Edited Jan 1, 2015 at 12:50 AM
I just started using spfspot. Works fine with a Raspberry Pi and my 5000TL-20 interverter. Thanks for a great piece of software!

I'm logging additional parameters via the donation mode on PVOutput.org, such as inverter temperature and AC voltage. One thing I noticed is that, first thing in the morning, the first few measurements do not have the additional parameters. Looking through the first full day's log after installing the software, I found:
[06:32:32] INFO: Uploading 18 datapoints, starting with 20141230,05:05,22454200,0 => OK (200)
That's the first entry in the log file for the day.

Sunrise is around 5 am (Brisbane) right now. So, the first upload moved 18 data points in a batch; thereafter, a single datapoint every 5 minutes. (I'm running SBFspot via cron every 5 minutes, 24x7.) Because things like voltage and temperature are spot values, they are missing from the first 17 data points.

I double-checked latitude and longitude, thinking that the sunrise calculation may be off. I have:
Latitude=-27.5
Longitude=153.0
which is right for the Brisbane area.

So, I upped the offset, hoping that this might cause the logging to kick in earlier:
SunRSOffset=3600
But, this morning, the first ten data points don't have the additional parameters either. The log shows as the first entry for the day:
[06:00:37] INFO: Uploading 11 datapoints, starting with 20150101,05:10,22535230,0 => OK (200)
even though the inverter turned on a 5:10 am.

My question is whether this is normal, or do I have a config error, or is there a bug somewhere?

For what it's worth, I've set up the SQL view to leave V5 as NULL (for Wunderground ambient temperature), V6 is used for AC voltage, and V7-V11 are used for VDC1, VDC2, PDC1, PDC2, and inverter temp.

Cheers,

Michi.
Jan 1, 2015 at 12:31 AM
Edited Jan 1, 2015 at 12:35 AM
Here are today's first 11 data points as recorded by PVOutput.org:
01/01/15    06:00   0.526kWh    0.080kWh/kW 1,248W  1,248W  0.189kW/kW  24.0C   244.4V  
01/01/15    05:55   0.422kWh    0.064kWh/kW 1,440W  1,440W  0.218kW/kW  23.9C   -   
01/01/15    05:50   0.302kWh    0.046kWh/kW 1,248W  1,248W  0.189kW/kW  23.7C   -   
01/01/15    05:45   0.198kWh    0.030kWh/kW 696W    696W    0.105kW/kW  23.6C   -   
01/01/15    05:40   0.140kWh    0.021kWh/kW 624W    624W    0.095kW/kW  23.5C   -   
01/01/15    05:35   0.088kWh    0.013kWh/kW 324W    324W    0.049kW/kW  23.4C   -   
01/01/15    05:30   0.061kWh    0.009kWh/kW 300W    300W    0.045kW/kW  23.4C   -   
01/01/15    05:25   0.036kWh    0.005kWh/kW 240W    240W    0.036kW/kW  23.4C   -   
01/01/15    05:20   0.016kWh    0.002kWh/kW 156W    156W    0.024kW/kW  23.4C   -   
01/01/15    05:15   0.003kWh    0.000kWh/kW 36W     36W     0.005kW/kW  23.4C   -   
01/01/15    05:10   0.000kWh    0.000kWh/kW 0W      -       -           23.4C   -
Coordinator
Jan 1, 2015 at 10:10 AM
Most likely this is caused by miscalculation of sunrise time
To verify sunrise time, add verbose logging (-v5)
-finq will force data retrieval before sunrise, so this is another option too
Jan 1, 2015 at 10:23 AM
I just tried with -v5 and get, at the end of the output:
Thu Jan  1 20:19:50 2015: INFO: Starting...
sunrise: 04:57
sunset : 18:44
So, it doesn't look like sunrise time is calculated incorrectly.

I'll add -finq and see what happens tomorrow morning.

Thanks for the quick reply, I appreciate it!

Cheers,

Michi.
Jan 1, 2015 at 6:07 PM
Edited Jan 1, 2015 at 7:56 PM
dear michihenning,
as you already checked, - the timing information, eg "sunrise-sunset" seems correct,

from your information
most elements that are related to the SBFspot - the part that retrieves information from inverter seems correct,

-to be 'more' certain about this you migth use the "csv-table" output as reference
you can make the csv output active (if not yet in use) by adapting the parameters in "SBFspot.cfg" file
all data, also used for the upload, will become visible in the csv output tables (spot & daily)
(activation of csv tables does not impact the sql data - neither the operation for upload)

note that a possible cause of the problems might be related with the fact that in your case the settings
for "brisbane" apply DLS switched "off" (which is different from Sydney/Melbourne . . .)
see the extract from the date_time-zone table
"Australia/Brisbane","EST","EST","","","+10:00:00","+00:00:00","","","","+00:00:00"
"Australia/Sydney","EST","EST","EST","EST","+10:00:00","+01:00:00","1;0;10","+02:00:00","1;0;4","+03:00:00"

it looks that for the interaction with pvoutput (for the added parameters)
you're interaction is treated different from "brisbane" timezone (but as being in 'sligthly' southern zone of sydney)

note to get full details when executing SBFspot - you can apply the options -d5 -v5
please redirect outputs via redirect instructions 1>>stdout.txt 2>>stderr.txt to capture the outputs

kr wim
Jan 1, 2015 at 9:48 PM
Thanks for the tips, kr wim!

This morning, I got the same behaviour, with the first upload being a batch. That's despite having added -finq last night to the cron job.

I've checked the timezone setting. It definitely is "Australia/Brisbane".

Looking at the csv output, The first few lines are:
sep=;
Version CSV1|Tool SBFspot3.0.3 (Linux)|Linebreaks CR/LF|Delimiter semicolon|Decimalpoint dot|Precision 3


;;;;Watt;Watt;Amp;Amp;Volt;Volt;Watt;Watt;Watt;Amp;Amp;Amp;Volt;Volt;Volt;Watt;Watt;%;kWh;kWh;Hz;Hours;Hours;%;Status;Status;degC
dd/MM/yyyy HH:mm:ss;DeviceName;DeviceType;Serial;Pdc1;Pdc2;Idc1;Idc2;Udc1;Udc2;Pac1;Pac2;Pac3;Iac1;Iac2;Iac3;Uac1;Uac2;Uac3;PdcTot;PacTot;Efficiency;EToday;ETotal;Frequency;OperatingTime;FeedInTime;BT_Signal;Condition;GridRelay;Temperature
02/01/2015 06:00:01;SN: 2100602564;SB 5000TL-20;2100602564;128.000;256.000;0.341;0.690;375.120;371.430;379.000;0.000;0.000;1.569;0.000;0.000;241.760;0.000;0.000;384.000;379.000;0.000;0.125;22570.674;49.920;9748.219;9605.357;76.863;Ok;Closed;53.890
Note that the first line of output is for 6 am, even though the first entry that is uploaded to PVOutput.org is for 5:05 am, and the first line with extended data is at 6 am. The first line in the log file shows that the initial upload was for a batch:
[06:00:24] INFO: Uploading 12 datapoints, starting with 20150102,05:05,22570549,0 => OK (200)
I've checked the time on the inverter and the Pi. They are both correct and within one second of each other. The Pi's time is set with ntp, and the SyncTime setting is set to 1 in SBFspot.cfg.

I'm wondering whether incorrect DST offset is responsible. In particular, the inverter's time zone also is Brisbane. Is it possible that, when SBFspot syncs the time with the inverter, the DST correction (or non-correction for Brisbane, rather) is being applied twice? In that case, when I look at the inverter's time setting with SunnyExplorer, I'd be masking the error because SunnyExplorer also sets the inverter time from the PC time.

I'll try logging with stdout and stderr redirected into separate log files. Will report back tomorrow :-)

Cheers,

Michi.
Jan 2, 2015 at 9:13 AM
Edited Jan 2, 2015 at 9:18 AM
dear michi, from your last info i'll assume that the problem is in the cronjob,

rational:
SBFspot creates several data tables: - spot table at each execution with the effective values from that moment
from the memory of the inverter SBFspot extracts
  • the daily file (energy-production with records per 5minutes)
  • the month file (energy-production AT END of completed day - one record for each day)
  • events file (info about events and so on) you can see these also with sunExplorer
from your info it looks that on your system
  • SBFspot is executed the first time at 06h00 in the morning then SBFspot extracts thet daily file for that day and the spot values for the moment
  • the subsequent step is upload function
    it reads the database and sends (not yet treated) records - in your case from sunrise up to 06h00 and the extended data for 06h00m
you can further control this by checking the time that the daily csv file and the spot csv file are created (see OperSystem tools)
  • if i'm right then the daily file and spot file are created at 06h00m
note: -the option -finq does not influence since it's use is to instruct SBFspot (eg for debugging) to execute even if its dark
dark for SBFspot is untill "sunrise minus sunRSoffset" (see sbfspot.cfg)
note: -if interogation of the inverter is stopped before sunset then it is good practice to execute SBFspot sith the option -ad2
this allows to be certain that all records of the perviousu day are treated as well

for further investigation - please provide with
your sbfspot.cfg
the stdout.txt and stderr.txt
the csv tables for daily and spot
info about sript / cronjob do you apply one job for SBFspot AND upload function ??

kr wim
Jan 2, 2015 at 9:54 AM
SillieWimons wrote:
dear michi, from your last info i'll assume that the problem is in the cronjob,
Aargh!!!

I'm sorry, my fault, I should have been more careful. I just checked the crontab entry and, indeed, the first run of SBFspot in the morning was at 6:00 am. I've set it to 5:00 am now, which I'm sure will fix the issue. I'll have to wait until tomorrow morning but, given what you said, I'm certain that will be the end of it.

My apologies for wasting your time, and thanks for your help and the explanation!

Cheers,

Michi.
Jan 2, 2015 at 9:58 AM
Edited Jan 2, 2015 at 9:58 AM
PS: One more thought. I created that crontab entry by copying and pasting out of the doc:

https://sbfspot.codeplex.com/downloads/get/901202

That example logs from 6-23. It might be better to change that to 4-22? Difficult to pick the right interval, given the different latitudes where people might be using the software. Maybe add a remark that the span should be a little larger than the earliest sunrise and latest sunset in summer?

Cheers,

Michi.
Jan 2, 2015 at 11:13 AM
Edited Jan 2, 2015 at 3:37 PM
dear michi, glad that it might be working

you're correct about the document but
all parameters and values, eg time-window for the cronjob are to be set for the individual installation,
the text in that doc does not yet include a "how-to" part to find these values but for a start
the values in the doc fit for most locations

note: i expect that sbf and snowmiss will be happy to get "usefull" input for improvements

as a rule of thumb
a simple approach is to use the sunrise / sunset timing for the longest day at your location (as you state)
  • those values can be found from the internet (wheather stations, .. ) they depend on your location and timezone (including the DLS setting)
apply the exact values rounded to 5minutes (hour/minutes), whether it is usefull to add time before sunrise / after sunset is personal viewpoint,
  • omit the usage of option -finq in the cron
  • SBFspot does calculate at each execution the daylight "time-window" for operation at your location
    the parameter "sunRSoffset" (see sbfspot.cfg) does increase this window with some minutes
  • by using this approach the real operation of SBFspot does follow the sun-evolution
    execution is skipped when "it's dark"
!!!! do not forget my remark on the option -ad2 for some installations (eg orientated to the west) there might be production after sunset

kr wim

if you do need to install more accurate crontasks ???
then you must setup several crontasks, each one covering a period in a day and an "active period" during the year
  • to start - you need one crontask that fits the day-period for the shortest day (eg 08h00-17h00)
  • add more tasks that cover step-by-step the daylength increases (eg from 21march untill 21september from 06h00--07h55 & 17h05--19h00)
Jan 2, 2015 at 10:13 PM
After fixing the crontab entry, things work perfectly, of course. My apologies for the noise and thanks again for your help!

Cheers,

Michi.