Daemon upload to many datapoints

Topics: 4. Bugs
Sep 16, 2015 at 2:22 PM
Hello,

Thanks voor de new version 3.1.1. I'm using it on a RPi since a couple of weeks.

Since more then a year I am used tot start de RPi every 3 hours, let it run for 5 minutes and than shut it down again. For me it worked fine.
Beginning this summer it was not working so smoothly anymore, maybe because of the size of the Sqlite3-database. The database was locked at times. The RPi was running with version 3.0.3.

I hoped that with version 3.1.1. the problems were over but stil it looks that there some problems whith the daemon.

Today, and also earlier, I found in the daemon logfile:
[14:58:45] INFO: : Starting... 
[15:01:36] ERROR: : Uploading 85 datapoints, starting with 20150916,08:00,8978651,0 Bad request 400: Batch size greater than 30 records [85] 
[15:03:32] ERROR: : Uploading 85 datapoints, starting with 20150916,08:00,8978651,0 Bad request 400: Batch size greater than 30 records [85] 
[15:05:29] ERROR: : Uploading 85 datapoints, starting with 20150916,08:00,8978651,0 Bad request 400: Batch size greater than 30 records [85] 
[15:07:26] ERROR: : Uploading 86 datapoints, starting with 20150916,08:00,8978651,0 Bad request 400: Batch size greater than 30 records [86] 
[15:09:23] ERROR: : Uploading 86 datapoints, starting with 20150916,08:00,8978651,0 Bad request 400: Batch size greater than 30 records [86] 
[15:11:19] ERROR: : Uploading 87 datapoints, starting with 20150916,08:00,8978651,0 Bad request 400: Batch size greater than 30 records [87] 
[15:13:18] ERROR: : Uploading 87 datapoints, starting with 20150916,08:00,8978651,0 Bad request 400: Batch size greater than 30 records [87] 
[15:15:15] ERROR: : Uploading 87 datapoints, starting with 20150916,08:00,8978651,0 Bad request 400: Batch size greater than 30 records [87] 
[15:16:09] INFO: : Stopping Daemon... 
[15:17:02] INFO: : Starting... 
[15:18:05] INFO: : Uploading 30 datapoints, starting with 20150916,08:00,8978651,0 => OK (200) 
[15:20:11] INFO: : Uploading 30 datapoints, starting with 20150916,10:30,8978792,120 => OK (200) 
[15:22:14] INFO: : Uploading 29 datapoints, starting with 20150916,13:00,8979289,216 => OK (200) 
[15:26:12] INFO: : Uploading datapoint: 20150916,15:25,8979952,524,,,,223.28,211.26,210.34,268,261,32.9 => OK (200) 
[15:32:00] INFO: : Uploading datapoint: 20150916,15:30,8979988,383,,,,222.65,201.91,205.11,197,191,33 => OK (200) 
[15:37:49] INFO: : Uploading datapoint: 20150916,15:35,8980023,393,,,,221.33,209.83,213.03,203,195,33 => OK (200) 
The RPi was started at 14:58, after that the daemon is trying to upload too many datapoints??? (The RPi was not switched off after 5 minutes.)

At 15:16 there was a reboot of the RPi and after that the daemon is working well.

Please advise.

Frans Schepers
Coordinator
Sep 16, 2015 at 8:53 PM
Daemon has not been changed in V3.1.1 - I'm testing optimized queries to improve performance.
For #datapoints, I suppose you have donated to PVO before, having the possibility to upload 100 datapoints at a time. Now your batch upload limit is back to 30, but due to bug 131 this limit is (was) wrong
Sep 26, 2015 at 7:27 PM
SBF, thanks for your reply. It solved a week ago my upload problem. I also donated again to PVO.
Now you introduced version 3.1.2 and the 131-bug is also fixed in the 'official' software. Thanks!

I hope you find faster queries for the daemon. My database is locked for more than 60 seconds at times and this time is increasing.