Missing records in daydata table

Topics: 4. Bugs
Apr 14, 2015 at 7:41 PM
Edited Apr 14, 2015 at 7:42 PM
Maybe this is related with table DayData not updated at the expected spot-time , but I'm not sure.

This week my upload of data to pvoutput already two times were blocked, because "Bad request 400: Power value too high for system size sbfspot".
A further analysis showed me that all the records were present in the table spotdata, but one was missing in daydata. The record following the missing record shows the double power and is rejected by pvoutput.
For example in spotdata I have the timestamps:
1429014603
1429014904
1429015203
1429015504
1429015803
1429016103
1429016404

And in daydata I have the records:
1429014600
1429014900
-> missing: 1429015200
1429015500
1429015800
1429016100
1429016400

If pvoutput rejects the record, I manually correct it by creating a new record with a new manually averaged "TotalYield" and "Power" an then the uploads continues.
But sometimes the data is within the range of pvoutput and being accepted, with a huge spike as result: pvoutput

I tried to see how the values in daydata are calculated (just in excel) not yet in the code. And I don't see a relation between the tables, while I'm sure the relation between the numbers exist.

Right now I'm running 3.03, so I don't if this is fixed in the latest beta. But I can't fiund any reference to this problem, so I expect not.

Thanks for this program, because I think it's very useful.

If you need any info, I can provide it.

Jeroen
Coordinator
Apr 15, 2015 at 6:49 AM
Hi Jeroen,
I doubt this is an SBFspot issue. Did you check it with Sunny Explorer?
Coordinator
Apr 18, 2015 at 8:20 PM
Edited Apr 18, 2015 at 8:27 PM
See Issue 105 for a workaround.
However, these workarounds don't eliminate the spikes as shown in your example.
Apr 18, 2015 at 9:21 PM
Edited Apr 18, 2015 at 9:38 PM
Right now I've checked the data with SunnyExplorer.

I'll show the data from the various sources round the spike:
First Sunny Explorer:
14-4-2015 17:00 4676,954 0,612
14-4-2015 17:05 4677,005 0,612
14-4-2015 17:10 4677,053 0,576
14-4-2015 17:15 4677,101 0,576
14-4-2015 17:20 4677,147 0,552
14-4-2015 17:25 4677,192 0,54
14-4-2015 17:30 4677,235 0,516

Then vwSpotdata:
14-4-2015 17:00 14-4-2015 17:00 SN: {SERIALNR} SB 1200 {SERIALNR} 828 0 5.752 227,1 0 144 0 775 0 0 3.414 0 0 227,1 0 0 828 775 93,6 5262 4676222 50,01 20550 18816,6 74,5 OK ? 0
14-4-2015 17:05 14-4-2015 17:05 SN: {SERIALNR} SB 1200 {SERIALNR} 869 0 6,17 227,4 0 141 0 814 0 0 3.582 0 0 227,4 0 0 869 814 93,7 5330 4676290 50,01 20550,1 18816,7 74,5 OK ? 0
14-4-2015 17:10 14-4-2015 17:10 SN: {SERIALNR} SB 1200 {SERIALNR} 847 0 6.009 226,8 0 141 0 793 0 0 3.498 0 0 226,8 0 0 847 793 93,6 5397 4676357 50 20550,2 18816,7 74,5 OK ? 0
14-4-2015 17:15 14-4-2015 17:15 SN: {SERIALNR} SB 1200 {SERIALNR} 819 0 5.731 225,2 0 143 0 767 0 0 3.408 0 0 225,2 0 0 819 767 93,7 5461 4676421 49,98 20550,2 18816,8 74,5 OK ? 0
14-4-2015 17:20 14-4-2015 17:20 SN: {SERIALNR} SB 1200 {SERIALNR} 810 0 5.831 227 0 139 0 759 0 0 3.346 0 0 227 0 0 810 759 93,7 5524 4676484 49,96 20550,3 18816,9 74,5 OK ? 0
14-4-2015 17:25 14-4-2015 17:25 SN: {SERIALNR} SB 1200 {SERIALNR} 780 0 5.654 226,3 0 138 0 731 0 0 3.234 0 0 226,3 0 0 780 731 93,7 5587 4676547 49,97 20550,4 18817 74,5 OK ? 0
14-4-2015 17:30 14-4-2015 17:30 SN: {SERIALNR} SB 1200 {SERIALNR} 758 0 5.303 226,2 0 143 0 710 0 0 3.139 0 0 226,2 0 0 758 710 93,7 5647 4676607 49,96 20550,5 18817,1 74,5 OK ? 0

At last vwDaydata:
14-4-2015 17:00 SN: {SERIALNR} SB 1200 {SERIALNR} 4676237 780 1
14-4-2015 17:05 SN: {SERIALNR} SB 1200 {SERIALNR} 4676303 792 1
14-4-2015 17:10 SN: {SERIALNR} SB 1200 {SERIALNR} 4676371 816 1
14-4-2015 17:20 SN: {SERIALNR} SB 1200 {SERIALNR} 4676499 1536 1
14-4-2015 17:25 SN: {SERIALNR} SB 1200 {SERIALNR} 4676559 720 1
14-4-2015 17:30 SN: {SERIALNR} SB 1200 {SERIALNR} 4676619 720 1

And now I see that the error already exists in Sunny Explorer, but because of the time-offset I looked at the wrong time:
14-4-2015 16:00 4676,237 0,78
14-4-2015 16:05 4676,303 0,792
14-4-2015 16:10 4676,371 0,816
14-4-2015 16:20 4676,499 1,536
14-4-2015 16:25 4676,559 0,72
14-4-2015 16:30 4676,619 0,72

Little to do about errors in the input. I'll try the workaround, but it won't help with these kind of spikes. :(

But my conclusion is that SBFspot is working fine.
Will try a firmware update tomorrow. Maybe this is solved.
Coordinator
Apr 19, 2015 at 6:01 PM
I had a closer look at the code and it's possible to solve it. I assume each interval is 5 minutes and miscalculate power accordingly.
I made a simulation on PVoutput with original and corrected data.
(There is a 1 Wh difference with your data due to some shift in the beginning.)
Coordinator
Apr 19, 2015 at 8:35 PM
Released V3.0.7 (beta) with a fix.
If it's ok, please tell us.
Marked as answer by SBF on 4/19/2015 at 1:35 PM