Data no uploading to PVoutput

Topics: 5. Support
Mar 11, 2015 at 6:46 PM
I ran SBFspot manual per instructions and it writes a row to vwspotdata each time I run it. I'm running it in the morning, 10AM here but I have to use -finq because it things it's dark out. My clock is set to UTC time, so 5:00PM, but have the timezone set in the cfg file.

When I run SBFspotUploadDaemon, nothing happens. It writes the following to the log.

tail -f SBFspotUpload20150311.log
[17:24:35] INFO: Starting SBFspotUploadDeamon Version 1.0.1
[17:24:35] INFO: Starting Daemon...

The problem is nothing is being uploaded to the PVoutput.org website.
Mar 11, 2015 at 8:30 PM
Maybe I don't understand how it works, but this is what's happening. If I leave the SBFspotUploadDaemon running, nothing ever gets uploaded. When I run SBFspot, it takes uploads the data and works fine. Do I need to put SBFspot on cron like very 5 minutes? I heard you don't need to do that with SBFspotUploadDaemon.
Coordinator
Mar 11, 2015 at 10:13 PM
SBFspot has to run every 5 minutes to fill your DB
Mar 12, 2015 at 1:18 PM
Edited Mar 15, 2015 at 6:28 PM
dear buellwinkle,
there will be a good reason for the fact that your system is set to UTCtime and not to your real local time

my proposal is that you run SBFspot maunally with the option -d5 -v5 -finq 1>stdout.txt included in the command-line
  • in the stdout.txt file you'll find the "time" values that are used by SBFspot
  • you might post the result for further investigation
be aware that that setting of your system clock has several impacts, (see below)

kr wim

-a- the "cron" instructions that you create for SBFspot (see documentation)
(remark also for SBFspotUploadDaemon)
do assume that your system clock is set to local time
hence the cron will look like 5 6-21 * * where 6-21 indicates the hour-window that tasks are executed
in your setup you might need (my guess 13-4) time-offset is 7hours

-b- ??? assuming that your inverter has a continious RTC-clock then WHEN
--AND in the cfg you did put the time-set flag to NO (otherwise your inverter time is set equal to system-time = UTC)
--AND in the cfg you use, for the spot table, "INVERTER" as time source
--AND the inverter time is set to local time (so you might need to adapt that time according DLS)
then i expect that PVOutput will accept your values

-c- SBFspot will calculate the sunrise and sunset time by using:
  • your system date
  • your system time (sbfspot extracts "system-info" such that it can apply eg DLS switch)
  • your position, cfr the values that you indicate in the cfg file
  • your time zone, cfr value in the cfg file
SBFspot compares your "system-time" with the calculated "sunrise" and "sunset" values
  • SBFspot assumes the "solar-active" window to be between "sunrise<->sunset"
  • in the cfg file "SunRSOffset" allows to extend that window (default 15min before and 15min after)
  • when your system time (which is at UTC) is outside the "solar-active" window then SBFspot announces "it's dark"
  • option -finq allows to bypass that control
Apr 21, 2015 at 1:41 PM
Hi,

I've just installed sbfspot on a raspberry pi and I also have problems uploading data to pvoutput. The SQlite db is populated fine (see later), however there must be something strange with time. In the smadata directory I get both a 2008 and a 2015 subdir. The 2008 dir only has the csv files, with the name corresponding to the date reported by the inverter (sma sb 4000); the 2015 dir has an Events subir with the events csv (named after the time as reported by the pi).

I have tried setting SynchTime to 0 or 1 and I don't see changes in the behavior. As reported by the OP, my uploader logfile only says

pi@raspberrypi /mnt/users/davide/pi/smadata/logs $ cat SBFspotUpload20150421.log
[14:23:41] INFO: Starting SBFspotUploadDeamon Version 1.0.1
[14:23:41] INFO: Starting Daemon...
[14:23:42] WARNING: iz4ugl_Bachelet is not yet member of SBFspot Team. Consider joining at http://pvoutput.org/listteam.jsp?tid=613
pi@raspberrypi /mnt/users/davide/pi/smadata/logs

I tried to capture output with SBFspot -d5 -v5 -finq 1>stdout.txt and I see that the time values are:

Timezone=Europe/Brussels
CalculateMissingSpotValues=1
DateTimeFormat=%d/%m/%Y %H:%M:%S
DateFormat=%d/%m/%Y
TimeFormat=%H:%M:%S
SynchTime=0
SunRSOffset=900
DecimalPoint=dot
[...]

End of Config

Tue Apr 21 14:31:32 2015: INFO: Starting...
sunrise: 06:22
sunset : 20:04
Connecting to 00:80:25:2E:F4:CE (1/10)
Initializing...
SUSyID: 125 - SN: 905477994 (0x35F87F6A)
getPacket(2)
[...]
<<<=================================>>>
INV_SWVER : '02.60.03.R' Sat Jun 28 02:08:35 2008
getInverterData(512)
[...]
MAX_pcktBuf is now 204 bytes
INV_NAME : 'SN: 2130292160' Fri Jun 27 21:35:53 2008
INV_CLASS : 'Solar Inverters' Fri Jun 27 21:35:53 2008
INV_TYPE : 'SB 4000TL-21' Fri Jun 27 21:35:53 2008
INV_TYPE : 'SB 4000TL-21' Fri Jun 27 21:35:53 2008
[...]
Tue Apr 21 14:31:35 2015: INFO: Done.

where the first and the last time refer to the PC time, the others refer to the inverter time. In the cfg file I have

OutputPath=/mnt/users/davide/pi/smadata/%Y
OutputPathEvents=/mnt/users/davide/pi/smadata/%Y/Events

which as I wrote seem to be expanded with different %Y. The sqlite db seems to be ok:

pi@raspberrypi /mnt/users/davide/pi/smadata $ sqlite3 SBFspot.db
SQLite version 3.7.13 2012-06-11 02:05:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> select * from vwspotdata;
2015-04-21 14:35:04|2015-04-21 14:35:00|SN: 2130292160|SB 4000TL-21|2130292160|1579|953|5.723|5.841|276.03|163.25|2450|0|0|10.464|0.0|0.0|234.22|0.0|0.0|2532|2450|96.8|18740|1605536|49.98|1795.84|1718.94|74.1|OK|Closed|45.1
2015-04-21 14:31:34|2015-04-21 14:30:00|SN: 2130292160|SB 4000TL-21|2130292160|1591|951|5.739|5.547|277.34|171.64|2461|0|0|10.624|0.0|0.0|232.67|0.0|0.0|2542|2461|96.8|18597|1605392|49.99|1795.78|1718.88|74.9|OK|Closed|45.1
2015-04-21 14:30:05|2015-04-21 14:30:00|SN: 2130292160|SB 4000TL-21|2130292160|1600|967|5.862|5.927|273.12|163.25|2483|0|0|10.679|0.0|0.0|232.56|0.0|0.0|2567|2483|96.7|18536|1605331|50.0|1795.75|1718.86|74.1|OK|Closed|45.6
2015-04-21 14:25:04|2015-04-21 14:25:00|SN: 2130292160|SB 4000TL-21|2130292160|1617|971|5.839|6.097|276.98|159.42|2502|0|0|10.768|0.0|0.0|232.34|0.0|0.0|2588|2502|96.7|18324|1605122|49.98|1795.67|1718.78|74.5|OK|Closed|46.0
2015-04-21 14:20:04|2015-04-21 14:20:00|SN: 2130292160|SB 4000TL-21|2130292160|1646|995|6.121|5.946|268.97|167.43|2554|0|0|11.037|0.0|0.0|231.47|0.0|0.0|2641|2554|96.7|18116|1604911|49.98|1795.59|1718.69|74.1|OK|Closed|46.3
2015-04-21 14:16:50|2015-04-21 14:15:00|SN: 2130292160|SB 4000TL-21|2130292160|1652|998|6.125|5.993|269.71|166.65|2564|0|0|11.01|0.0|0.0|232.87|0.0|0.0|2650|2564|96.8|17978|1604773|49.98|1795.53|1718.64|74.5|OK|Closed|46.3
sqlite>

Any hints? Thanks.
Apr 21, 2015 at 2:18 PM
Self-replying to report that the problem of the two log directories, one with the date of the inverter, and the other with the date of the PC, has been fixed setting

CSV_Spot_TimeSource=Computer

in the cfg file. However, data is still not being uploaded to pvoutput.

Is there any debug setting I can use to understand what 's going on?

Thanks,
Davide
Coordinator
Apr 21, 2015 at 2:38 PM
Your inverter has a wrong date/time (See INV_NAME : 'SN: 2130292160' Fri Jun 27 21:35:53 2008)
In config, you should have SynchTime=1 to set it right. If this doesn't work, you'll have to adjust it manually with Sunny Explorer
Have a look in vwPvoData, I'm sure all dates are in 2008. You can only upload 14 days back.
The best way is to start with a brandnew db, as all dates are wrong
Apr 21, 2015 at 3:52 PM
Many thanks, updating date/time with Sunny Explorer fixed it (just setting SynchTime=1 did not), I now see data being uploaded to pvoutput.

Davide
Apr 25, 2015 at 8:54 PM
Edited Apr 28, 2015 at 9:54 AM
dear davide,

you reported that "SynchTime=1" did not solve the time setting problem for your type of inverter
it is known that the "SBxx00 TL 2x" series of inverters will start the clock at 01jan2008, at commisioning
  • according your info, (i assume that) your inverter has been started 6months ago ?? PLEASE CONFIRM !!
Note: as long as the Inverter Date-Time was not set correctly -> READ (the inverter-clock was NOT set to your local time)
  • executing SBFspot (which uses the default options -ad1 -am1) will receive from the inverter a reply 'NO-DATA"
    for Month file and Daily file
  • your spot data file will have 2008june27.spot as date indicator
    ?? PLEASE CONFIRM !! you can check this, if you did not omit the csv option, then you will find in your output path for .csv files:
    20080627-spot aso for the spot files
you will not find the month files NOR daily files (they have the date indications mentioned below)
since SBFspot does ask for 2015april and 2015april21
200806 month files
20080627 aso for the daily files

note: if i'm wright then -> you might recuperate the old month files
-a- execute sbfspot once, having csv output active, with the option -am92
sbfspot -d5 -v5 -am92 -ad0 -ae0 -finq -nosql 1>catchmont-ou.txt 2>catchmont-er.txt

to recuperate the daily files is more tricky -> you must use a system that is set to "2008june30"
-b- this is tricky -> eg changing date in sunny explorer system to 30june2008 (sun explorer might force inverter time)
-c- alternate - execute sbfspot on your PC (change the PC time to 2008june30) with Synchtime = 0
sbfspot -d5 -v5 -am0 -ae12 -ad200 -finq -nosql 1>catchday-ou.txt 2>catchday-er.txt

good luck :-)

can you provide additonal information that might allow to identify these "limits"
thus one might be able to preprare a test-setup to analyse in detail what is needed for this

examples of info:
your config file before and after
the stdout.txt files from your tests (please indicate "step-by-step" when the files where created)

thank you for providing the extra info

kr wim
May 9, 2015 at 3:01 PM
Dear Wim,
you reported that "SynchTime=1" did not solve the time setting problem for your type of inverter
it is known that the "SBxx00 TL 2x" series of inverters will start the clock at 01jan2008, at commisioning
  • according your info, (i assume that) your inverter has been started 6months ago ?? PLEASE CONFIRM !!
That is correct. I got my inverter about 6 months ago, but only very recently I started to collect data directly from it. So I basically do not have previous files, I just bumped into the problem of log files having the wrong date when I installed sbfspot, and that was because the inverter date was never set. Once I set it with Sunny Explorer everything was fine.

This is why I cannot provide you with the answers you were asking me -- I do not have any "wrong" log files anymore, since I basically managed to fix the problem in a couple of days, sorry about that.

Many thanks for your help anyway!

Davide
May 11, 2015 at 2:13 PM
Edited May 11, 2015 at 2:15 PM
dear davide, it's fine that your system works now,

may i although ask you to contribute to the project,

if i'm wright -> then your inverter might still have the old month files,

even if that info is not of direct interest to you
we'd like to understand what was done by the (software) of your inverter after you did modify the date-time

for that reason, can you execute the below command and inform us on the result
with in the config.file: -allow csv output -appply a correct path for the output of the csv files

CSV_Export=1 <- the default is 1
OutputPath=C:\SBFSpot\catchmont <-this for windows for linux depends on your setup
OutputPathEvents=C:\SBFSpot\catchevents

-a- execute sbfspot to inspect whether the inverter has still "month"files
the option -nosql will avoid that there is output to your database
sbfspot -d5 -v5 -am92 -ad0 -ae0 -finq -nosql 1>catchmont-ou.txt 2>catchmont-er.txt

rational: a large number of "new" or "candidate" users of sbfspot might experinece the same problem as you had,
since inverter not set to "local" time
your experience might help to complement our info and improve the instrctions

after analysis of the result it might be that we need also the "archive events"
-b- execute sbfspot to inspect for the "user" events
sbfspot -d5 -v5 -am0 -ad0 -ae92 -finq -nosql 1>catch-U-events-ou.txt 2>catch-U-events-er.txt

-c- execute sbfspot to inspect for the "installer" events
sbfspot -d5 -v5 -am0 -ad0 -ae92 -installer -password:1111 -finq -nosql 1>catch-I-events-ou.txt 2>catch-I-events-er.txt
May 11, 2015 at 3:52 PM
Hi,

you can fetch a tar.gz file with the output of the commands you mentioned above from https://www.dropbox.com/s/omfd0gcxqbzi979/sbfspot_debug.tar.gz?dl=0. I hope it is useful, let me know if I can help in any other way.

Cheers,
Davide
May 11, 2015 at 10:24 PM
Edited May 12, 2015 at 1:56 PM
dear davide, thanks a lot for that information

I'll include:
  • my, first and quick, analysis below
  • demand for some additional info
    kr wim

??? to make live easier for further analysis
-> it would help if you can send a copy of the two events .csv files (-user -installer)
reason - the contents of events-records is complex, so SBFspot does not decode it in stdout for events
the available alternative -> consult events via the csv files, this is straight forward
it might be that there is a 2015 and a 2008 directory with .csv files

??? the inverters reaction on the DATE set instruction allows to make the assumption that
  • as I did suggest earlier "repeat it might be" -> but all depends on the inverters behaviour
    that the daily file tables - records of production with 5minute interval (prior to 21april) are still in your inverters memory,
    and might show up - consistent (see my notes about the month file - below)
you can check this with the command
sbfspot -d5 -v5 -am0 -ad70 -ae0 -finq -nosql 1>catchdays-ou.txt 2>catchdays-er.txt
these .csv files will be visible in your directory OutputPath=/mnt/users/davide/pi/smadata/%Y
if a 2008 directory should appear, please inform

!!!!! in view of issue c108 you might need to use SBFspot version v305 for this


Analysis:
-a- about the "month" data -> from what is directly visible

as a result of your Date-Time Change command (using SunExplorer) at 21apr2015 around 14h37m
-a.1- the inverter's internal database has been reorganised
-a.2- the records / date (in the month files) remain a continious range - and on first glance it is consistent

-> the "Date" gap for "day/day records between:
-a.4- start-up of your inverter (OLD internal time 01jan2008 ) till (OLD internal-time 11june2008)
AND
-a.5- reset of date - to 21apr2015 is eleminated

-b- about the EVENT tables - user and installer that DATE gap is maintained in the inverters data
-b.1- events were added to the list from (OLD) 1jan2008 -> (OLD) 11june2008
-b.2- after 21apr2015 (local-time) around 14h37m the new events are added to the apr2015 file and so on
note: the records of the events keep the original Date-Timestamps
see event records with a timestamp from 01jan2008 till 11june2008