This project has moved and is read-only. For the latest updates, please go here.

Beginner's questions: how and when to run SBFspot under windows?

Topics: 5. Support
Sep 19, 2014 at 11:42 AM
Hi,

I read the quick reference PDF and got SBFspot to run with my two sunny boy inverters on a windows PC with a very cheap USB-BT-Stick. I now get data in the SQLite db and also the .csv files. Great!

I did not yet try the upload service.

Things I wonder about: why is the upload implemented as a service, but the download not? Hard to access BT from a service?

How often and when and with which parameters should I run it? I don't see any parameters to have the tool download all new data since previous run. Am I missing something? Do I have to specify a few days to download just to be sure?
Sep 19, 2014 at 1:09 PM
Edited Sep 19, 2014 at 2:56 PM
maf_soft wrote:
I did not yet try the upload service.
Don't wait too long, you can only go back for 14 days
Things I wonder about: why is the upload implemented as a service, but the download not? Hard to access BT from a service?
The download could run as a service too, but it's still like in the early days: a simple command line tool. Version 4 maybe.
How often and when and with which parameters should I run it? I don't see any parameters to have the tool download all new data since previous run. Am I missing something? Do I have to specify a few days to download just to be sure?
It depends of what you wan to do.
In my case, storage in SQLite an upload to PVO: Every 5 minutes default run (-ad1 -am1 -ae1)
This means I download every 5 mins all data for current day/month. I know it's overkill, but it won't hurt. It goes fast and I don't want to create a bunch of cron jobs (scheduled tasks)

I"m sure Wim S. will give you some more info on this.
Sep 19, 2014 at 1:39 PM
Edited Sep 19, 2014 at 2:09 PM
SBF wrote:
In my case, storage in SQLite an upload to PVO: Every 5 minutes default run (-ad1 -am1 -ae1)
This means I download evert 5 mins all data for current day/month. I know it's overkill, but it won't hurt.
Funny that we are thinking about it so differently :)

A way to synchronize only new/missing data was the first thing I would expect from such a tool, and I was very surprised it isn't there...

Even though it is fast, it doesn't feel good for me, to download a lot of unnecessary data. I am always trying to optimize things.

And there are also real issues with your approach: it could happen that the server is down for a while, and one day is not enough. Then I always have to remember this and manually change the parameters. Or one day it is shut down for maintenance 1 hour before sunset. Then this hour is missing, because on the next day (or after sunset) this data isn't fetched.
Or people just use their normal PC for this, which doesn't run every day. They could even decide not to create any scheduled task and just let it run once at startup. Downloading let's say 14 full days every time is not so fast.

So, we have a feature request :) my suggestion is to have the parameters -ad, -am and -ae only specify the maximum number of days/months, but it should automatically be reduced to fetch only data that is newer than existing data in the db.
It goes fast and I don't want to create a bunch of cron jobs (scheduled tasks)
I don't understand what you want to say with that...? Why would you need a bunch of tasks?

-Moritz
Sep 19, 2014 at 2:55 PM
And there are also real issues with your approach: it could happen that the server is down for a while, and one day is not enough. Then I always have to remember this and manually change the parameters. Or one day it is shut down for maintenance 1 hour before sunset. Then this hour is missing, because on the next day (or after sunset) this data isn't fetched.
That again depends... running SBFspot every 5 mins will store all your data in the DB, while the uploader will fail due to PVO down. When PVO gets up again, all missing data will be uploaded. However, when your RPi or PC goes down (crash) you have a problem.

But I totally agree there is still room for improvement, so feel free to raise a feature request. I know already a few people who will vote for it ;-)
I don't understand what you want to say with that...? Why would you need a bunch of tasks?
1 job to fetch the day data (every 5 mins)
1 job to fetch month data (once a day, after sunset)
1 job to fetch events (once a week)
1 job to fetch spot data (every minute)

Be aware SQL DB support has started with V3. Before that, there was no 5min data upload, only spotdata
Sep 19, 2014 at 3:02 PM
SBF wrote:
But I totally agree there is still room for improvement, so feel free to raise a feature request. I know already a few people who will vote for it ;-)
Ok, will do that.
1 job to fetch the day data (every 5 mins)
1 job to fetch month data (once a day, after sunset)
1 job to fetch events (once a week)
1 job to fetch spot data (every minute)

Be aware SQL DB support has started with V3. Before that, there was no 5min data upload, only spotdata
Seems I will have to find out what spotdata is about :) - maybe current counters at time of download?
Sep 19, 2014 at 3:08 PM
That's it! spot data is real-time data
Hence the name SBFspot ;-)
Sep 19, 2014 at 9:34 PM
Edited Sep 19, 2014 at 9:50 PM
hej maf_soft, I'll give some complement information about planning and scheduling the tasks

first some basic info on what SBFspot is able to do and what is not possible (due to limits of the inverter)
then some recommondations on task planning
!!! It did take time to step-step construct the correct planning

BASIC info
SBFspot is a great tool to capture the "SPOT" data from your inverters, -> your remark on "current counters at time of download" is very correct
since you have already the .csv files you can consult the various types of data that is in the "spot" table.
Since the inverters themselves do not keep a history of this (spot) information, the only way to create a database or table with
that information is by repeated execution of SBFspot.

Which history information is available ??? -> an Inverter keeps in its memory data records for:
-the yield with a 5minute interval -> this is the daily file :-) the command to read that table is the -ad## get archive of daily file for ##days
-the yield at "end" of a day - note that (in reality) this value is only added to the table at the start-up on the next day -> this info goes into the monthly file :-) the command to read these table is the -am## get archive of monthly file for ##months
-the events (user - installer) -> this is the the event-user and event-installer file :-) the commond to read these table si the -ae## info for ##days
!!! for the time being this is all that we are aware of -vav inverters
!!! once the SMA EnergyMeter can be read via SBFspot then extra info might appear

IMPORTANT: some inverters (the ones that have not a build-in communication module) will only write history in a correct way to the tables,
if they receive each day again a day-time indication - if not the daily file and the "end-day" record will be overwritten (they get lost) by the next day info

depending on the amound of data already stored in its archive, - an inverter will keep
  • the monthly file for at least 30 months
  • the daily file for more then 130days
  • the events for ??
ABOUT YOUR TASK PLANNING
consider that all your tasks will be run in a cycle per 5minutes - only one task at a time can grap the communication to the inverter
since you have two inverters you will need MIS to access them both or otherwise you must define two cycles (the tasks in these two cycles should not interfer)
that cycle will be the planning assist to insert tasks into Task-Scheduler of MS-Windows (below a suggestion)

-a- 1 job to fetch the day data (every 5 mins) be aware that CPU-clock =/= inverter clock
--- use the command -ad1 -am0 -ae0 -sp0
--- plan this task to be run at the 30th sec of Minute 0 - then there is no collision on the access of your BT-stick

-b- 1 job to fetch month data (once a day, DURING the day, "after sunset" - it's dark) execute this task eg. three times a day at 10h00 at NOON at 15h00
--- use the command -ad4 -am2 -ae5 -sp0 (by using -ad4 you will complete the daily files of the preceeding days -am2 idem for monthly file
--- skip SPOT values since your other task (see -d-) will run each minute
--- plan this task to be run at the 30th sec of Minute 1 - then there is no collision on the access of your BT-stick

-c- 1 job to fetch events (once a week) obsolete one better treat this as part of the previous task

-d- 1 job to fetch spot data (every minute)
--- use the command -ad0 -am0 -ae0
---plan this task to be run at the 00sec of a minute
---be aware that the task scheduler for MS-Windows has limits (depending on the version of the OS it might be that you need 5 or even 10 tasks to run a complete cycle of tasks

-e- if you intend to fetch spot data more often then each minute then you should first "plan" the cycle, such that all your tasks fit in the plan
for instance - i do fetch the spot data at a rate of each 15sec - to do this i have added extra jobs that run on minute 0, 1, 2, 3, 4 of my cycle
the tasks of type a and b need to fit in the plan so at minute 0 - second30 I'll run in sequence the task for daily and spot

-f- be aware - you might need to create "supervision" and control tasks - these ones control whether the spinning wheel run smoot and correctly
eg - in my plan fits at minute 0 - second 20 a sanity function if the system is stuck then the sanity function will
reboot my PC after it has stored all data needed to know what happened
eg - in my plan fits a "wake-up" job and a "hibernate" task" my system is asleep at night and during several periods in the day that the spot data is less interesting
eg - there is the upload to PVoutput and it is only done once per 30minutes
eg - there is a task (once aday) that writes day/day the sunrise - sunset information that is calculated by SBFspot into a table - in the same table I add the inverter-time and the PC-time, (rational - the time source for my inverter comes from an independent machine)
eg - there is a task that reads the .csv files and merges them with other information (a tool from ron)

the little bit of tasks above correspond in my system with 5d + 5e + 1b + 1a + 5 varia = total 17 different task - they all fit into the spinning wheel
note that each task has a dedicated set of settings

kr wim
Sep 22, 2014 at 8:11 AM
SillieWimons wrote:
!!! It did take time to step-step construct the correct planning
Thanks a lot! I will look at the data and decide if I need spot data.

For calling every few seconds I would recommend not to use scheduled tasks, but let a batch file run in a loop with sleep in between. I can send an example if you like.