PV Output consumption data

Topics: 1. General
Jul 30, 2015 at 8:24 AM
Hello to all, this is my first posting.
I have been uploading to PV Output for some years, using "PV Bean Counter" but I am interested in moving to a Pi-based solution.
I have set up SBFspot and it is working fine but I would now like to add consumption data. Currently I use an "Owl" USB monitor but I also have a Current Cost version. I found the pvccupload software (predecessor to SMAspot?) and installed ccterm, which reads the CC monitor and writes the data to a text file - seems to work OK. Now comes the question!

Reading other posts on here, I think I have gathered that the SMAspot Upload Daemon can upload all the PVO data, including consumption providing it is in the sqlite database table - is that correct? So, if I can make the necessary changes to the database and come up with a system to populate the correct fields, would that work? - or am I over-simplifying!

I see there is a good deal of interest in using the SMA Energy Meter as a source of consumption data, but this is a very expensive piece of kit compared to Owl or CC - you need to generate a lot of solar kWh to pay for one of those!

Thanks for any info!
Coordinator
Jul 30, 2015 at 9:17 AM
Yes, that's the way it should work...
Populating the consumption table - preferable every 5 minutes.
The vwPVOdata view will combine the generation/consumption and the daemon uploads it all together. Didn't try it yet, so please go for it ;-)
Jul 30, 2015 at 9:49 AM
SBF - thanks for your quick response! You mention "consumption table" - is that a separate table in the sqlite setup? Any info on this and how it maps to PVO would help get me up the learning curve. As soon as I get a minute, I'll have a look at the existing database used by SBFspot.
Jul 30, 2015 at 11:42 AM
SBF - ignore my last question, I think I understand it now. I'll let you know if I make any progress!
Jul 30, 2015 at 1:49 PM
hej penman

pvccupload was a previous -independent- project from ron patton

with the project ron created an environment where various data-sources (each generating .csv type files)
could be (intelligent) combined to one output
that output (.csv file) was subsequent uploaded to PV Output (or other . . . )

ron patton was one of the early adapators to migrate to SBFspot for reading information from his inverters

as you did mention:
  • several users of SBFspot are interested in tools to include consumption data in the upload platform
  • current cost is a good (cheaper & more functions) alternative vav SMA's energy meter,
    with "current cost" you will effective measure consumption per "circuit"
note: there is a recent tool to capture "consumption data" -> see "SMAPPEE"
according the technical data it is an indirect measuring tool - it delivers information about a lot of equipment

please keep us posted with the progress of your set up

kind regards wim
Jul 30, 2015 at 3:09 PM
Edited Jul 30, 2015 at 3:11 PM
Hi wim,

The ccterm application component of pvccupload could easily modified to update a database. I was thinking about do that but it is not on my radar at the moment. I do keep up and run the latest release of SBFspot and have it update the SQLite DB just for the fun of it :)

Cheers,
Ron

P.S. The pvccupload application environment with SBFspot has been running flawlessly for over two years which is one of the reasons I have not been in a rush to convert pvccupload to a database application.
Jul 30, 2015 at 5:18 PM
Hi Wim and Ron,
Many thanks for your replies, I am new to all this and this info helps me get into it. Of course there are easier ways to present this data, "Owl Intuition" and as Wim mentioned, SMAPPEE. But I like PVOutput so I would like to stay with that. PV Bean Counter was a great project, but sadly the developer seems to have abandoned it, he hasn't been heard of for more than a year and updates and bug fixes are badly needed. I hope the guy is OK! The Pi is a very good choice for this application - apart from being cheap to buy, it has very low power consumption so it fits well with a project aimed at reducing energy consumption - you don't want to put solar panels on the roof and then use 20% of their output to keep a power-hungry PC going 24/7 just to monitor their performance!

So, good work so far, I for one am most grateful and I'll be sure to let you know how I get on.

Cheers,
Joe.
Jul 31, 2015 at 9:53 AM
hej penman (joe) -> happy to give you the information

for the record, i did mention SMAPPEE as tool because it offers an alternate solution for monitoring a complete household
rational
  • when using "current cost" as tool, one needs a clamp for each circuit and depending on the installation a number of transmitters
  • the clamps provided by current cost are fit for "glaobal" installation" measurements
    -- (the upper limit for Iac is 100A = thus 100 * 230V = 23kWatt)
    -- (the lower measurement for Iac is 50mA = thus 0,05*230V = 12Watt) consumption under 6watt not measured; 7--> 18Watt ~= 12; and so on
    -- it is a PITTY that "Current Cost" does not have a clamp for the range 10mA (>3Watt) to 20A (4,5Kw)
  • to measure energy consumptions below 500Watt one must use the IAMs
  • SMAPPEE does an analysis of the consumption using a kind of "footprint" for each device,
    -- so using one clamp you measure several devices
as indicated by ron his latest version (now 24months ago) still runs without problems -> but this uses .csv files

be aware: the approach with .csv files is different since
-- sbfspot delivers values via .csv file output,
-- pvccupload treats the results from the various inputs (sbfspot & ccterm & . . . . )
with pvccupolad -> the pvoutput deamon is not used (done via pvccupload) (ron to correct me if wrong :-? )

kr wim
Jul 31, 2015 at 12:54 PM
Hi, Yes, that is correct. Stick figure diagram: ccterm and SBFspot output to txt/csv data collector files. Pvccupload reads these files every five minutes, performs data munging and then sends the results to pvoutput.org ccterm ----> ccterm.txt | ccterm
----> sensor.txt |---------> pvccupload ----------> pvoutput.org SBFspot ---> daily.csv | 73 Ron / W4MMP On 7/31/2015 05:54, SillieWimons wrote:
Aug 1, 2015 at 11:19 AM
Hi Wim and Ron,

You raise some interesting issues. As I understand it, these inductive current clamps are all pretty inaccurate below a couple of hundred watts. For a domestic installation, an upper limit of 23kW is very unlikely to be a problem, but I must say, it would be nice to have accurate readings in the 0 to 500W region - this is where I think we waste a lot of energy. Never mind, it should give comparative figures, even if they aren't strictly accurate.

Back to pvccupload, etc. Wim says above "-- sbfspot delivers values via .csv file output". That's interesting, I thought it gave its output via database only. I shall go and look for the .csv file. If I have understood Ron correctly, he is using ccterm and pvccupload to send consumption data to PVO and also SBFspot to deliver the generation data - have I got that right? If so, that's a quick (if slightly "dirty"!) solution, which is known to work.

As a matter of interest, where are you guys based? I am in the UK and I gather our wiring regulations for grid-tied solar installations are a bit different to the US versions. Hmm, having said that, it may just be terminology which differs - can you post pictures on here? If so, I'll put up sketches of the two most common UK wiring schemes.

Regards,
Joe.
Aug 1, 2015 at 3:23 PM
Hi Joe,
If you would like, take a look at: http://sourceforge.net/projects/pvccupload/files/. This where the pvccupload project is now located. There are documentation files there. One of the best doc files is the sensor power point presentation. It does into some detail regarding power configurations and how pvccupload interacts with ccterm and SBFspot.

The application suite only runs on Linux and has only been tested on the Raspberry PI running Raspbian. You will need a moderate level of expertise in Linux to be able to configure the application suite. The suite was originally created to support US power configurations as all of the US uses a " 240V (or 208V) split phase" power configuration. Over the course of time, the suite was modified to handle non-US power configurations.

As has been mentioned, I no longer actively support pvccupload. However if you decide to try your hand at it, I will help out as time permits with the applications themselves. Figuring out Linux is up to you.
73
Ron / W4MMP
On 8/1/2015 07:19, penman wrote:

From: penman

Hi Wim and Ron,

You raise some interesting issues. As I understand it, these inductive current clamps are all pretty inaccurate below a couple of hundred watts. For a domestic installation, an upper limit of 23kW is very unlikely to be a problem, but I must say, it would be nice to have accurate readings in the 0 to 500W region - this is where I think we waste a lot of energy. Never mind, it should give comparative figures, even if they aren't strictly accurate.

Back to pvccupload, etc. Wim says above "-- sbfspot delivers values via .csv file output". That's interesting, I thought it gave its output via database only. I shall go and look for the .csv file. If I have understood Ron correctly, he is using ccterm and pvccupload to send consumption data to PVO and also SBFspot to deliver the generation data - have I got that right? If so, that's a quick (if slightly "dirty"!) solution, which is known to work.

As a matter of interest, where are you guys based? I am in the UK and I gather our wiring regulations for grid-tied solar installations are a bit different to the US versions. Hmm, having said that, it may just be terminology which differs - can you post pictures on here? If so, I'll put up sketches of the two most common UK wiring schemes.

Regards,
Joe.

Aug 1, 2015 at 5:51 PM
Hi Ron,
Thanks for that. I don't actually have a moderate level of Linux expertise, but I'm in the learning curve and it's gradually getting better! I did succeed in getting ccterm working but the documentation refers to "smaspot" as the source of generation data, not SBFspot and I had assumed that a major change from the sma_ to the SBF_ version was the move from csv text file to database. I also have SBFspot working (that was comparatively easy, even for me) but I can't see any .csv or .txt files being generated, so I hadn't proceeded to getting pvccupload itself working. smaspot is not included in the downloads on sourceforge.net (unless I've missed something) so I wasn't able to set that up.

If you are telling me that SBFspot can work with pvccupload, that's great, but I might need a little guidance to get it working - at least to know which files need to be in which directories and if there is any additional configuration necessary. I can generally figure these things out once I have a few clues!

I did make use of the documentation you provided with pvccupload, I would certainly have been lost without it. And once I found the copyit and makeit files, I was able to see what directories to create and I was in business. As a newcomer, it can be a bit painful to pick your way through this, but you learn a lot more quickly than having someone saying "type this"!

It sounds as if our 240v single phase domestic supply is a bit simpler to set up for PV input and monitoring. 3 phase supplies to domestic properties are very rare in the UK, so mostly you just have one wire to worry about, neutral being bonded to earth (ground) anyway. PV can be fed in via a spare breaker in the distribution box ("consumer unit"), making it very difficult to monitor consumption. Or the PV can have its own single-position consumer unit, connected to the mains supply and the main consumer unit via a "Henley Block", making it dead easy to monitor consumption. This is the setup I have.

Regards,

Joe.
Aug 1, 2015 at 6:52 PM
Hi Joe,
SMAspot equals SBFspot. They are identical. SMAspot was the previous name before SBF needed to change to avoid copyright infringement issues with SMA. SBFspot has always generated csv files and in fact csv files were the only output it generated. SBF added database output later on but kept it backwards compatible with previous releases.

Take a look at the sample SBFspot.ini file in there you will see:
# CSV_Export (default 1 = Enabled)
# Enables or disables the CSV Export functionality
CSV_Export=1

This will turn on csv functionality. As for file locations, simply make sure everything (data files) points to the same directory. On my system everything points to:
/home/ron/sq1/datafiles. Of course, you can choose any location you like. And then, application log and error files point to /home/ron/sq1/logs.

SBFspot is not included with the pvccupload package. It needs to be downloaded from the codeplex website.

"It sounds as if our 240v single phase domestic supply is a bit simpler to set up for PV input and monitoring. 3 phase supplies to domestic properties are very rare in the UK, so mostly you just have one wire to worry about, neutral being bonded to earth (ground) anyway. PV can be fed in via a spare breaker in the distribution box ("consumer unit"), making it very difficult to monitor consumption. "
Which is exactly why I created pvccupload. I made a huge mistake when in the planning of my solar system when we had our house built (a bit over 3 years ago). The solar system ties in via a breaker in the distribution box making the determination of consumption a big pain in the butt. But re-doing the mains would have been an even bigger pain.


Hope this helps,
73
Ron / W4MMP
On 8/1/2015 13:52, penman wrote:

From: penman

Hi Ron,
Thanks for that. I don't actually have a moderate level of Linux expertise, but I'm in the learning curve and it's gradually getting better! I did succeed in getting ccterm working but the documentation refers to "smaspot" as the source of generation data, not SBFspot and I had assumed that a major change from the sma_ to the SBF_ version was the move from csv text file to database. I also have SBFspot working (that was comparatively easy, even for me) but I can't see any .csv or .txt files being generated, so I hadn't proceeded to getting pvccupload itself working. smaspot is not included in the downloads on sourceforge.net (unless I've missed something) so I wasn't able to set that up.

If you are telling me that SBFspot can work with pvccupload, that's great, but I might need a little guidance to get it working - at least to know which files need to be in which directories and if there is any additional configuration necessary. I can generally figure these things out once I have a few clues!

I did make use of the documentation you provided with pvccupload, I would certainly have been lost without it. And once I found the copyit and makeit files, I was able to see what directories to create and I was in business. As a newcomer, it can be a bit painful to pick your way through this, but you learn a lot more quickly than having someone saying "type this"!

It sounds as if our 240v single phase domestic supply is a bit simpler to set up for PV input and monitoring. 3 phase supplies to domestic properties are very rare in the UK, so mostly you just have one wire to worry about, neutral being bonded to earth (ground) anyway. PV can be fed in via a spare breaker in the distribution box ("consumer unit"), making it very difficult to monitor consumption. Or the PV can have its own single-position consumer unit, connected to the mains supply and the main consumer unit via a "Henley Block", making it dead easy to monitor consumption. This is the setup I have.

Regards,

Joe.

Coordinator
Aug 1, 2015 at 7:17 PM
Hi Ron,
I couldn't have explained it better ;-)
Thanks
Aug 2, 2015 at 8:46 AM
Hi Ron,

Haha! now I'm beginning to understand. Well now that I know what's going on, I think my first move will be to set up pvccupload as you have explained. I think I'll add a test system to my existing PVO account so that my existing set-up can continue undisturbed for the time being. I won't get much time this week, so it may be a little while before I get this done. I may get an hour this morning though, so you never know, I might make some progress. I'll let you know....

As originally installed, my PV system did connect via a spare breaker in the main consumer unit, but I bit the bullet and re-wired it a couple of years ago. It was a pain to do, but I'm glad I made the effort. More recently I've been using a device ("iBoost") to divert unused PV energy to a water heater. Question is, do you count that as consumption or not? At the moment I do, but since even in our climate, this can use 5 or 6 kWh per day, it can be 30% of total consumption. Hmm, I might re-think that one.

Regards,

Joe.
Aug 2, 2015 at 10:49 AM
Hi Ron,

I am having to pack up for now, but I've made a bit of progress. However, generation data is not being displayed on PVOutput. Here is a fragment of the pvccupload logfile, I'm hoping you can put your finger on the reason for the WARNING - see below:


^[[1;32;40mSun Aug 2 11:24:11 2015: INFO: ^[[0mAlternate Power Mapping is Disabled

^[[1;32;40mSun Aug 2 11:24:11 2015: INFO: ^[[0mPvccupload has started and initialized. Version: PVCCUPLOAD:Version 3.2.0 COMPILE DATE:Aug 2 2015
^[[1;32;40mSun Aug 2 11:24:11 2015: INFO: ^[[0mSunrise: 05:21 Sunset: 20:45

^[[1;32;40mSun Aug 2 11:24:11 2015: INFO: ^[[0mRecovery Routine - Starting
^[[1;32;40mSun Aug 2 11:24:11 2015: INFO: ^[[0mCurl Command - GET STATUS: curl -q -m 5 -s -S --stderr - -d "d=20150802" -d "t=10:45" -d "ext=1" -H "X-Pvoutput-Apikey: f8994d9a5fb92049868445ed05aaa6
87f1691c88" -H "X-Pvoutput-SystemId: 39698" http://www.pvoutput.org/service/r2/getstatus.jsp
^[[1;32;40mSun Aug 2 11:24:11 2015: INFO: ^[[0mPVOutput GET STATUS Response:20150802,10:45,NaN,0,234,1419,NaN,NaN,0.0,NaN,NaN,NaN,NaN,NaN,NaN,


^[[1;32;40mSun Aug 2 11:24:14 2015: INFO: ^[[0mIn Recovery Mode - Ignoring Sunrise-Sunset Times for Generated Power Values
^[[1;32;40mSun Aug 2 11:24:14 2015: INFO: ^[[0mCalling Main Routine
^[[1;32;40mSun Aug 2 11:24:14 2015: INFO: ^[[0mSOLAR STATUS(POWER): -1.00 TIME: 02.08.2015 10:50
^[[1;32;40mSun Aug 2 11:24:14 2015: INFO: ^[[0mCC STATUS(POWER): 1293.07 TIME: 02.08.2015 10:50
^[[1;33;40mSun Aug 2 11:24:14 2015: WARNING: ^[[0mDATE NOT FOUND in Current SMA-SPOT File. Is it Cloudy?: Setting Solar Power to Zero
^[[1;32;40mSun Aug 2 11:24:14 2015: INFO: ^[[0mCC File Read Success.
^[[1;32;40mSun Aug 2 11:24:14 2015: INFO: ^[[0mPower Correction Factor Applied: 1.00 Corrected CC Power Value: 1293.07
^[[1;32;40mSun Aug 2 11:24:14 2015: INFO: ^[[0mFormula Number: 10 Name: RAW CC POWER (NO CALCULATIONS APPLIED Power Values: Sma-spot: 0.00 | CC: 1293.07 | Power Used: 1293.07 | Total Extended Par
ameter Watts: 0.00

You can see the upload at:
http://www.pvoutput.org/intraday.jsp?id=5386&sid=39698

Many thanks for your help!

Regards,

Joe.
Aug 2, 2015 at 8:48 PM
Hi Joe,

What comes to mind is the output format of the SBFspot csv data records. I have attached a copy of my SBFspot configuration file. Let's see if codeplex allows attachments. Please check the output record format of SBFspot, it must look like this:

sep=;
Version CSV1|Tool SBFspot3.0.8 (Linux)|Linebreaks CR/LF|Delimiter semicolon|Decimalpoint dot|Precision 3

;Squireoaksfarm;Squireoaksfarm
;SB 4000US;SB 4000US
;2002038573;2002038573
;Total yield;Power
;Counter;Analog
dd.MM.yyyy HH:mm:ss;kWh;kW
02.08.2015 06:10:00;20619.583;0.000
02.08.2015 06:15:00;20619.583;0.000
02.08.2015 06:20:00;20619.583;0.000
02.08.2015 06:25:00;20619.584;0.012
02.08.2015 06:30:00;20619.585;0.012
02.08.2015 06:35:00;20619.588;0.036
02.08.2015 06:40:00;20619.591;0.036

If I remember correctly (I would need to take a look at the code), pvccupload reads past the header lines and then scans for the current 5 minute reporting period. Pvccupload really does run without error, but it is not forgiving of data that is not formatted properly. I did not put a lot of time into coding for things such as incorrectly formatted data as these things rarely change. SBF follows the software engineer's hippocratic oath: first do no harm :-) . I did however, put a lot of time into making sure the applications handle processing errors such as network timeouts and the like. To date, I have never had the ccterm or pvccupload "crash". (Knock on wood).

FYI: When / if you need to send the output of pvccupload via email, I would suggest XXXXXX(ing) out your pvoutput API key. If the wrong person gets a hold of it, that person can make a real mess of your data on pvoutput.org.

73
Ron / W4MMP
On 8/2/2015 06:49, penman wrote:

From: penman

Hi Ron,

I am having to pack up for now, but I've made a bit of progress. However, generation data is not being displayed on PVOutput. Here is a fragment of the pvccupload logfile, I'm hoping you can put your finger on the reason for the WARNING - see below:


^[[1;32;40mSun Aug 2 11:24:11 2015: INFO: ^[[0mAlternate Power Mapping is Disabled

^[[1;32;40mSun Aug 2 11:24:11 2015: INFO: ^[[0mPvccupload has started and initialized. Version: PVCCUPLOAD:Version 3.2.0 COMPILE DATE:Aug 2 2015
^[[1;32;40mSun Aug 2 11:24:11 2015: INFO: ^[[0mSunrise: 05:21 Sunset: 20:45

^[[1;32;40mSun Aug 2 11:24:11 2015: INFO: ^[[0mRecovery Routine - Starting
^[[1;32;40mSun Aug 2 11:24:11 2015: INFO: ^[[0mCurl Command - GET STATUS: curl -q -m 5 -s -S --stderr - -d "d=20150802" -d "t=10:45" -d "ext=1" -H "X-Pvoutput-Apikey: f8994d9a5fb92049868445ed05aaa6
87f1691c88" -H "X-Pvoutput-SystemId: 39698" http://www.pvoutput.org/service/r2/getstatus.jsp
^[[1;32;40mSun Aug 2 11:24:11 2015: INFO: ^[[0mPVOutput GET STATUS Response:20150802,10:45,NaN,0,234,1419,NaN,NaN,0.0,NaN,NaN,NaN,NaN,NaN,NaN,


^[[1;32;40mSun Aug 2 11:24:14 2015: INFO: ^[[0mIn Recovery Mode - Ignoring Sunrise-Sunset Times for Generated Power Values
^[[1;32;40mSun Aug 2 11:24:14 2015: INFO: ^[[0mCalling Main Routine
^[[1;32;40mSun Aug 2 11:24:14 2015: INFO: ^[[0mSOLAR STATUS(POWER): -1.00 TIME: 02.08.2015 10:50
^[[1;32;40mSun Aug 2 11:24:14 2015: INFO: ^[[0mCC STATUS(POWER): 1293.07 TIME: 02.08.2015 10:50
^[[1;33;40mSun Aug 2 11:24:14 2015: WARNING: ^[[0mDATE NOT FOUND in Current SMA-SPOT File. Is it Cloudy?: Setting Solar Power to Zero
^[[1;32;40mSun Aug 2 11:24:14 2015: INFO: ^[[0mCC File Read Success.
^[[1;32;40mSun Aug 2 11:24:14 2015: INFO: ^[[0mPower Correction Factor Applied: 1.00 Corrected CC Power Value: 1293.07
^[[1;32;40mSun Aug 2 11:24:14 2015: INFO: ^[[0mFormula Number: 10 Name: RAW CC POWER (NO CALCULATIONS APPLIED Power Values: Sma-spot: 0.00 | CC: 1293.07 | Power Used: 1293.07 | Total Extended Par
ameter Watts: 0.00

You can see the upload at:
http://www.pvoutput.org/intraday.jsp?id=5386&sid=39698

Many thanks for your help!

Regards,

Joe.

Aug 2, 2015 at 9:37 PM
Hi Ron,
I'll have a better look at this in the morning (it's bedtime here!). But two things are obviously different in the SBFspot output. First I have version 3.0.3 and you have 3.0.8. Second, my date format is dd/MM/yyyy and yours is dd.MM.yyyy - I don't know if that's causing the date error.

I can't see your attachment, but I'm looking at this on my ipad which is a bit inconvenient and I may be missing it. If I can find a PM facility I'll send you my email.

I'll be in touch in 8 hours or so!

Regards,

Joe
Aug 2, 2015 at 9:52 PM
Hi Jo, Yep, that is the issue. The data record format must be dd.mm.yyyy (not the "/" version). As mention previously, that is controlled in the SBFspot initialization file. Here is a snippet of the initialization file with the correct entries: # DateTimeFormat
(default %d/%m/%Y %H:%M:%S) # For details see strftime() function # http://www.cplusplus.com/reference/clibrary/ctime/strftime/ #NOTE: there is a space between the %Y and %H parameters. DateTimeFormat=%d.%m.%Y %H:%M:%S # DateFormat (default %d/%m/%Y) DateFormat=%d.%m.%Y
# DecimalPoint (comma/point default comma) DecimalPoint=point # TimeFormat (default %H:%M:%S) TimeFormat=%H:%M:%S I think I was using the "." format long before SBFspot was available and I was using SMA's Sunny Explorer for capturing the solar power. So when
I moved to using SBFspot I changed from the the default "/" format to the "." format Make the above changes and you should be good to go. 73 Ron / W4MMP On 8/2/2015 17:37, penman wrote:
Aug 2, 2015 at 11:02 PM
dear joe, i did follow the interactions with ron and sbf

my below comments are about
-a- SMAspot <-> SBFspot
-b- major changes from SBFspot 2xx -> 3xx
-c- about measuring power: production - consumption
-g- who is where

kind regard wim

-a- as highlighted by ron SMAspot == SBFspot
the change of name was due to "avoid copyright infringement issues with SMA" -> SMA == SMA company

-b- version 3xx did delliver -> two major changes in the functions of SBFspot

-b.1- "reading data" from the inverter went seperated from "sending data" to 'example' PVoutput
the first function continued as "sbfspot.exe" the second function is "uploaddaemon"
sbfspot.exe does write information to a "desired" output format

-b.2- support for data export towards a "database" was improved -> and maintaining the support for .csv files
since version 30x "sbfspot.exe" can deliver the data in at least three formats (there is also support for: -123solar, -. . . )
-b.2-a- .csv file format prime objective is to be === with SMA's output format cfr sunny explorer
the command line option -nocsv competely inhibits csv output
several options in the config file allow to adapt: -formats; -header-seletc; -coding (EU<->US); -file-naming; -directories, . . .
-b.2-b- database type sqlite -> see cfg for options on target output
the command line option -nosql competely inhibits SQL output
-b.2-c- database type MySQL -> see cfg for options on target output
the command line option -nosql competely inhibits SQL output

-c- about measuring power production <-> consumption
major difference between "SMA-EnergyMeter" and "CurrentCost" & "Smappee" & . . .
-c.1- the capability to identify the direction of the energy flow "consume" <-> "produce"
but in most installations this is no problem if one does measure the "Iac"s at multiple positions
-c.2- to gather not only "current"="Iac" but also "tension"="Vac"

rational:
in most cases (nearly100%) one can distinguish in "electric" installation the circuits in three groups
-c.4- ONLY consumption
-c.5- ONLY production (PV installation)
-c.6- in most cases the "MAINS" is the only pro/sumer circuit
Energy send-to / received-from "MAINS" is the difference between "ONLY cons" <-> "ONLY prod"

Putting a "measuring" "Iac" clamp on the "MAINS" will not render full information
(except when using SMA's energy meter)
by selecting the measuring points (this means that de-facto you need at least 2 measuring clamps)
you can get all "energy" information (with a delta for not having thhe Vac measured)

in my environment i needed:
-f.1- clamp for PV-production
-f.2--5- clamps for all group circuits of consumption
-f.3- using the Uac from SBFspot's output

-g- i'm living in europe - belgium - single phase connected principles of connections in belgium
-g.1- null wire =/= ground
-g.2- null wire and phase/line is delivered by the electric network
in Belgium a single phase does provide two wires from the elctric network NULL and LINE and local delivered MY-GROUND
in Belgium a 3phase network does provide 4wires from the electric network NULL and P1, P2, P3 and local MY-GROUND
a delta in current between "NULL, LINE (or P1,P1,P3)" (is fail to GROUND) will cause a shut-off
Aug 3, 2015 at 9:51 AM
Hi Wim and Ron,

Once again, thank you for your help and explanations, it has got me going much quicker!

Yes, the date format was the problem with uploading generation data, that is now working fine as can be viewed at:
http://www.pvoutput.org/intraday.jsp?id=5386&sid=39698

The Current Cost monitor is generally reading a bit lower than my Owl monitor. Again, I'll watch that for a few days and see if there is a significant difference - if so, I guess I'll have to see if I can figure out which is right (if either!).

I was a bit surprised at the voltages showing on PVO, but now I realise that it is 'Udc1' so it makes sense! I may change that to 'Uac1' which is more meaningful. My panels are organised in two groups, 8 facing East and 12 facing West - Udc1 relates to the West-facing array which is only just starting to wake up this morning. By comparison, the East array is up to 430v. I found it a lot easier to load the SBFspot upload file into Excel to view that data!

My plan now is to let this run for a few days to convince myself that it's reliable and working as I want it to, and then I'll probably switch over to using this for my main PVOutput uploader. You can view my "old" system here:

http://www.pvoutput.org/intraday.jsp?id=5386&sid=4275

I need to give some thought as to how best to make the switch without losing data, but it should be easy enough.

I do still have another question, but it should be dead easy for you! This comes back to my inexperience with Linux - can you suggest suitable entries for rc.local to start ccterm and pvccupload on boot? I don't think I understand how that works properly yet!

Wim - it sounds as if your single phase supply is exactly the same as in the UK, though the distribution within the house is different. When I said there is only one wire to worry about I really meant just the one "live" wire - there is of course a neutral as well. Although neutral is bonded to earth/ground at the point of supply, the return is via the neutral only and any live/neutral imbalance or earth leakage will trip a protective circuit breaker at (typically) 30mA. I heard this morning from the electricity company that they are going to install a "smart meter" for both electricity and gas on 8th October - I hope this doesn't interfere with my wiring scheme!

I will also investigate additional probes for the CC monitor - I am thinking about monitoring the water heater consumption separately, but that's another story!

Kind regards,

Joe.
Aug 3, 2015 at 11:49 AM
great Joe,

I did purchased the raspberry pi few days ago right after I came to know the website of SBFspot (I was looking something to read and upload PV data on the web). I never heard before about rasberry pi but I followed the instructions and in few hours the raspberry pi was uploading the data on PVoutput (that for me was increbible)... Right after the happiness for that I came to same question of your (which I posted on an italian forum): due that PVoutput allows for accounting the energy consumption, how to manage it with raspberry in conjuction with SBFspot? I really need it in order to activate a/c or heating system (now manually, later via raspberry pi) in order to avoid giving energy to national network for free during the day (whilst have to pay for it during the night, at least so it is in italy). The point is that I'm not familiar at all with Linux and raspberry pi.

I have to questions:
1) do you mind to summarize the steps you followed to get the consuption uploaded over PVoutput
2) which meter did you use (cause I understand that Current Cost is a brand or am I mistaking)?

thanks and ciao,
gabriele
Aug 3, 2015 at 12:14 PM
Hi Joe,
I examined both of your pvoutput.org sites and I'm a bit confused. They both show "power used". I thought that previously to using pvccupload, you were not sending "power used" values. I'm sure I missed something in this thread, but would you clarify , please.
73
Ron / W4MMP
On 8/3/2015 05:51, penman wrote:

From: penman

Hi Wim and Ron,

Once again, thank you for your help and explanations, it has got me going much quicker!

Yes, the date format was the problem with uploading generation data, that is now working fine as can be viewed at:
http://www.pvoutput.org/intraday.jsp?id=5386&sid=39698

The Current Cost monitor is generally reading a bit lower than my Owl monitor. Again, I'll watch that for a few days and see if there is a significant difference - if so, I guess I'll have to see if I can figure out which is right (if either!).

I was a bit surprised at the voltages showing on PVO, but now I realise that it is 'Udc1' so it makes sense! I may change that to 'Uac1' which is more meaningful. My panels are organised in two groups, 8 facing East and 12 facing West - Udc1 relates to the West-facing array which is only just starting to wake up this morning. By comparison, the East array is up to 430v. I found it a lot easier to load the SBFspot upload file into Excel to view that data!

My plan now is to let this run for a few days to convince myself that it's reliable and working as I want it to, and then I'll probably switch over to using this for my main PVOutput uploader. You can view my "old" system here:

http://www.pvoutput.org/intraday.jsp?id=5386&sid=4275

I need to give some thought as to how best to make the switch without losing data, but it should be easy enough.

I do still have another question, but it should be dead easy for you! This comes back to my inexperience with Linux - can you suggest suitable entries for rc.local to start ccterm and pvccupload on boot? I don't think I understand how that works properly yet!

Wim - it sounds as if your single phase supply is exactly the same as in the UK, though the distribution within the house is different. When I said there is only one wire to worry about I really meant just the one "live" wire - there is of course a neutral as well. Although neutral is bonded to earth/ground at the point of supply, the return is via the neutral only and any live/neutral imbalance or earth leakage will trip a protective circuit breaker at (typically) 30mA. I heard this morning from the electricity company that they are going to install a "smart meter" for both electricity and gas on 8th October - I hope this doesn't interfere with my wiring scheme!

I will also investigate additional probes for the CC monitor - I am thinking about monitoring the water heater consumption separately, but that's another story!

Kind regards,

Joe.

Aug 3, 2015 at 12:15 PM
Oh, I will reply off thread regarding the scripts needed to automatically start pvccupload at boot time.
73
Ron / W4MMP
On 8/3/2015 05:51, penman wrote:

From: penman

Hi Wim and Ron,

Once again, thank you for your help and explanations, it has got me going much quicker!

Yes, the date format was the problem with uploading generation data, that is now working fine as can be viewed at:
http://www.pvoutput.org/intraday.jsp?id=5386&sid=39698

The Current Cost monitor is generally reading a bit lower than my Owl monitor. Again, I'll watch that for a few days and see if there is a significant difference - if so, I guess I'll have to see if I can figure out which is right (if either!).

I was a bit surprised at the voltages showing on PVO, but now I realise that it is 'Udc1' so it makes sense! I may change that to 'Uac1' which is more meaningful. My panels are organised in two groups, 8 facing East and 12 facing West - Udc1 relates to the West-facing array which is only just starting to wake up this morning. By comparison, the East array is up to 430v. I found it a lot easier to load the SBFspot upload file into Excel to view that data!

My plan now is to let this run for a few days to convince myself that it's reliable and working as I want it to, and then I'll probably switch over to using this for my main PVOutput uploader. You can view my "old" system here:

http://www.pvoutput.org/intraday.jsp?id=5386&sid=4275

I need to give some thought as to how best to make the switch without losing data, but it should be easy enough.

I do still have another question, but it should be dead easy for you! This comes back to my inexperience with Linux - can you suggest suitable entries for rc.local to start ccterm and pvccupload on boot? I don't think I understand how that works properly yet!

Wim - it sounds as if your single phase supply is exactly the same as in the UK, though the distribution within the house is different. When I said there is only one wire to worry about I really meant just the one "live" wire - there is of course a neutral as well. Although neutral is bonded to earth/ground at the point of supply, the return is via the neutral only and any live/neutral imbalance or earth leakage will trip a protective circuit breaker at (typically) 30mA. I heard this morning from the electricity company that they are going to install a "smart meter" for both electricity and gas on 8th October - I hope this doesn't interfere with my wiring scheme!

I will also investigate additional probes for the CC monitor - I am thinking about monitoring the water heater consumption separately, but that's another story!

Kind regards,

Joe.

Aug 3, 2015 at 12:27 PM
Hi Jo,
Ok, I just did a better job of reading the posts ! :-( . There appears to be something amiss with the pvccupload configuration. "Power used" and "Power generated" are tracking too closely and it appears that pvccupload has sent several instances of zero (0) power generated. Please take a look at the pvccupload log files for the times of zero power generated. Pvccupload is pretty good at letting you know what is going on.
73
Ron / W4MMP
On 8/3/2015 05:51, penman wrote:

From: penman

Hi Wim and Ron,

Once again, thank you for your help and explanations, it has got me going much quicker!

Yes, the date format was the problem with uploading generation data, that is now working fine as can be viewed at:
http://www.pvoutput.org/intraday.jsp?id=5386&sid=39698

The Current Cost monitor is generally reading a bit lower than my Owl monitor. Again, I'll watch that for a few days and see if there is a significant difference - if so, I guess I'll have to see if I can figure out which is right (if either!).

I was a bit surprised at the voltages showing on PVO, but now I realise that it is 'Udc1' so it makes sense! I may change that to 'Uac1' which is more meaningful. My panels are organised in two groups, 8 facing East and 12 facing West - Udc1 relates to the West-facing array which is only just starting to wake up this morning. By comparison, the East array is up to 430v. I found it a lot easier to load the SBFspot upload file into Excel to view that data!

My plan now is to let this run for a few days to convince myself that it's reliable and working as I want it to, and then I'll probably switch over to using this for my main PVOutput uploader. You can view my "old" system here:

http://www.pvoutput.org/intraday.jsp?id=5386&sid=4275

I need to give some thought as to how best to make the switch without losing data, but it should be easy enough.

I do still have another question, but it should be dead easy for you! This comes back to my inexperience with Linux - can you suggest suitable entries for rc.local to start ccterm and pvccupload on boot? I don't think I understand how that works properly yet!

Wim - it sounds as if your single phase supply is exactly the same as in the UK, though the distribution within the house is different. When I said there is only one wire to worry about I really meant just the one "live" wire - there is of course a neutral as well. Although neutral is bonded to earth/ground at the point of supply, the return is via the neutral only and any live/neutral imbalance or earth leakage will trip a protective circuit breaker at (typically) 30mA. I heard this morning from the electricity company that they are going to install a "smart meter" for both electricity and gas on 8th October - I hope this doesn't interfere with my wiring scheme!

I will also investigate additional probes for the CC monitor - I am thinking about monitoring the water heater consumption separately, but that's another story!

Kind regards,

Joe.

Aug 3, 2015 at 12:48 PM
Hi Jo, The proof positive method for determining the amount of power used is to attach a current meter to the mains (your friendly local electrician will have one ). It is a simple device that clamps onto the cable just as the power meter probes do. Measure
the current at one minute intervals starting on a five minute mark and record the measurements. Then take the average. Multiple that value times the mains voltage (which should be very consistent) and that is your "power used" averaged over five minutes. (Power
= Current * Voltage) 73 Ron / W4MMP On 8/3/2015 05:51, penman wrote:
Aug 3, 2015 at 1:23 PM
Hi Ron,

I have always sent both consumption and generation figures up to PVO but there are several reasons why I wanted to change to using the Pi - not the least being its own very low power consumption compared to a PC!

I'll have another look at the graphs on PVO, but my consumption will often track the generation, especially in the mornings. I use an "iBoost" device to divert surplus PV energy to the water heater (immersion heater). This cuts in, starting at about 40W when there is a surplus of about 200W and diverts power up to a maximum of 1kW. When the tank is hot, it cuts out. On sunny days, it usually claims at least 3kWh - but it can be 6kWh, depending on how much hot water has been used. This is just an attempt to export less!

I will look into the zero power generated though, that can't be right.

As to the power readings reported by the CC monitor, I suspect the difference is because on the Owl, the mains voltage is configurable and I have it set to 245v - our average. I'm not sure what voltage the CC meter is set up for, but I suspect it might be 230v. I don't know if it's configurable, it isn't mentioned in the booklet! At best though, these are VA meters, they take no account of power factor - I know in theory it should be close to 1.0 but we rarely use electric heaters these days, so most loads involve a motor or a transformer. My AC theory is a bit too rusty to discuss that in much detail! The difference between the two meters seems to be less than 5% so I'll monitor it for a few days before I take any action.

Thanks again for the help, I'm pleased with progress!

Joe.
Aug 3, 2015 at 1:33 PM
Hi Gabriele,

First of all, please don't think I'm an expert! Two weeks ago my knowledge of Linux was zero, so at best I'm a few days ahead of you in the learning process!

I am actually in the process of documenting what I did for my own purposes, but if you like, I'll tidy it up a bit and post it on here - give me a day or two to do this. It would probably then be helpful if one of the real experts would comment and correct it, then you should have a reliable installation and set up guide. I'm glad you've got SBFspot working, that's a great starting point.

I have used the "Current Cost EnviR" monitor for my usage data. In the UK, some of the energy companies like Eon have been giving them away to customers, so they then appear on eBay, pretty cheap. You will need to make sure you get the version with USB output.

Kind regards,

Joe.
Aug 3, 2015 at 1:35 PM
Hi Joe, I can tell you know your stuff :-) . I know that the U.S. version of the CC monitor has the capability of configuring the voltage. This is necessary as the U.S. has two voltage variants (208 and 240). When I get a moment I will check the documentation
for my monitor. 73 Ron / W4MMP On 8/3/2015 09:23, penman wrote:
Aug 4, 2015 at 11:28 AM
Edited Aug 4, 2015 at 11:32 AM
Hi Joe,

quote
I am actually in the process of documenting what I did for my own purposes, but if you like, I'll tidy it up a bit and post it on here - give me a day or two to do this.
unquote

it would be great! (I did the same on an italian forum after I got SBFforum to work)

Two questions for the experts:

1) from the data I can see on pvoutput (http://www.pvoutput.org/list.jsp?id=43368&sid=39616) I noticed that the step reading value is 12 W (I mean production can be 12W, 24W, 26W; not 6, 8, 9 W,....): I can imagine is a question of converting the analogue value into fixed string of bit, but could be reduced the step reading value (in the inverter or in the config file)?

2) Current Cost seems a product specifically designed for the intent of monitoring consumption and costs by itself and not sure is suitable for italian market too. I would prefer a energy meter like EASTRON SDM 220 (http://www.flanesi.it/blog/2015/06/25/contatore-eastron-sdm220-modbus-per-monitoraggio-energetico/), which better measures W than only A; an italian guy developed a tool to read from it with raspberry pi and analyse data in conjuntion with sw like 123Solar and MeterN (see http://www.flanesi.it/blog/2014/09/15/monitoraggio-energetico-con-raspberry-123solar-e-metern/). The question is: is it possibile to joint a device like EASTRON SDM 220 or similar with raspberry pi into PVoutput?

thanks und ciao,
gabriele
Aug 4, 2015 at 6:12 PM
dear fellow guys, some reactions (from wim) about the consumption / production items

-a- about gabriele demand about the 12Watt step in the values (for the daily values),
the WATT shown in the SMA inverter output is NOT-MEASURED but
unfortunate a calculated value: (powerafter minus powerbefore) divided timeperiod (kWh-kWh) / time
if you knowi that power is in kWh and period is 5minutes or "1/12" of an hour, then -> this explains the standard step of 12Watts
in fact the "attribute" Watt" can be discussed (it is used by SMA) -> it is kWh (per 5minute period) -> =~~ average Watt during 5minutes (rounded to 12)

-b- about gabriele information for alternate tools - GREAT info thank you
to join its info -> if the system delivers output in .csv files then it might easy match with ccterm

-c- about "DirectCurrent" power measuring - the calculation is simple:
Pdc = Idc * Udc (for that reason SBFspot can execute the calcul) if current is negative the result is negative

-d- about "AlernateCurrent" power measuring - for info to joe and gabriele
Pac =/= Iac¨* Uac the phase (°) between current and tension has an impact on that result
-d.1- in fact - at MAINS interface point
-d.1-c-> equal phase is energy flows out of Mains == consumption
-d.1-p-> opposite phase is engeryg flow towards MAINS (network) == production (by your PVplant)
therefore
-d.2- ALLWAYS DISTINGUISH whether your circuits are production <-> consumption
-d.3- if you only measure the current-value you can not deduct what position you are
-d.4- in general: "measuring current at MAINS is NOT USEFULL information,
since Iac on its own, (without phase) does not inform whether the circuit is in consume<->produce mode

-e- about the tension OR usage of a fixed value to recalculatie current -> power for info to joe
-e.1- from my long term measurements on my mains circuits
the delta on MY mains voltage is limited (231,5volt in average) -1,2% <-> +3,1%
minimum value (winter - 228,7volt) - highest (summer - midday - full production by all PVplants 238,1V)
-e.2- for calculating power (sbfspot delivers the Pac value for the PVplant)
-e.2-f- either one uses a fixed value (in my case 231,5) or
-e.2-m- the value measured by sbfspot values ) unless absent

-f- about tension/power from my knowledge pvccupload includes options to make calculations ??? ron
-f.1- if wright - one can calculate or recalculate the power values using pvccupload
but the correction might be limited -> in my case -1,5% <-> +3,%

-g- abut electric wiring scheme
-g.1- "Joe stated: "Although neutral is bonded to earth/ground at the point of supply,"
my note: in belgium the principle schem is that "neutral" and "line" (or "neutral" and "3*phase) deliver the power
the neutral line is not bound to the ground at "household" environment but at "point of distribution" (eg a power distribution post)
a small tension between neutral and ground (<<5Volts) can occur due to imbalance in power distribution
the differential-circuit breaker will detect a delta between neutral <-> line which is a flux into ground

kr wim
Aug 4, 2015 at 7:45 PM
The "mains" power value as reported by the CC meter may be adjusted by pvccupload. The value in the configuration file default is: 1.0 which means no adjustment. Use a value greater than 1 to increase the reported power and a value less than 1 to reduce
reported power). Ex: CC mains power * 1.1 (Ex: 1000W * 1.01 = 1010W) or Ex: (1000W * .99 = 990W). This performed before any subsequent processing. One of the features I like the best (what can I say, I wrote it :) ) is the ability to replay all the data for
the past 90 days. So say that you find that the CC meter is not reporting mains power properly over the course of a month or so. Pvccupload may stopped and the initialization file changed to modify the correction factor. Pvccupload then may be restarted at
a point of 30 days ago and it will replay the data with the new correction value. The raw data files are never modified. All power calculations are performed on the "fly". NOTE: You must be in pvoutput.org "donation mode" to replay 90 days worth of day. I
forget what the limit is for non donation mode. 73 Ron / W4MMP On 8/4/2015 14:12, SillieWimons wrote:
Aug 5, 2015 at 9:50 AM
Hi Guys,

As you may have noticed, I have been having a problem with zero generated power and zero dc volts being reported on occasions. I am not sure what is happening here, however I have found that my bluetooth "dongle can be a little intermittent, so it may be a hardware problem. If I find time, I'll go and buy a new one today and see if it helps. Just at the moment it seems to be working but it died at around 18:30 yesterday and stopped reporting until I had a fiddle with it this morning. If anyone has any other ideas though, please let me know! :-)

Wim - sorry, I may have used sloppy terminology. I didn't mean in the UK neutral and earth are bonded in the home, it's at the sub-station in the distribution network, same as for you and probably the same throughout Europe. And yes, AC is very inconvenient to work with! It can be very difficult to be sure which way the current is flowing, especially with small currents and unknown power factor.

Ron, thank you for the info on power correction in pvccupload. I will monitor it for a while longer and then decide if it is worth correcting. My present impression is that the CC meter agrees with my older Owl meter within about 2% and I've a feeling it may not be realistic to measure it more accurately than that. I think I've a current clamp which came with my "Fluke" multimeter. I've never used it but that may be more accurate and may be useful for calibration. I'll have a look for it. One other point, the PVOutput display generated by pvccupload is 5 minutes later than that generated by my old system, in other words the generation figure for say 8:00 on my old system is the same as 8:05 on pvccupload. It doesn't bother me at all, but I'd just like to be sure that is how it's meant to work.

Gabriele - I'm afraid I know nothing about the Eastron energy meter you mention. Assuming it can output data by bluetooth/USB or whatever, it should be possible to adapt ccterm to read it, though it would obviously involve some work. I can't see any reason why Current Cost shouldn't work for you and it's a fairly cheap, ready-made solution. It's always interesting to hear of these devices though, they are becoming more and more common with domestic energy management. Also, I'm conscious that I promised a description of how I installed and configured pvccupload - I haven't forgotten, but it may take longer than I expected. My son is getting married at weekend and as you may imagine, I'm not short of things to do! And that's what I must do now!

Regards to all,

Joe.
Aug 5, 2015 at 1:13 PM
Quick update. I changed the USB Bluetooth dongle at 12:35pm today. Hence the zero volts event at that time. Since then everything has been OK.

Hmm, confession time! I broke the golden rule of trouble-shooting and made two related changes at the same time. I installed the new bluetooth dongle and I moved the Pi a lot closer to the inverter to improve BT signal strength. If it is now OK, I'll never know which change made the difference! Looking on the bright side, if it's not OK, I'll have eliminated two possible causes! I'll give an update tomorrow.

Regards,

Joe.

http://www.pvoutput.org/intraday.jsp?id=5386&sid=39698
Aug 5, 2015 at 6:59 PM
Edited Aug 5, 2015 at 8:06 PM
dear joe about BTdongles the list of "trusted" types is short but "eilas" it is the reality

for BT transmission - two items are important:

-a- the experienced distance between "inverter" and "receiver" should be as limited as possible

-a.1- be aware: your inverter includes a "class1" sender/receiver (if you own a type with external antenna then be happy)
as a result the communication quality depends on the qualtity/type of "YOUR" bt-dongle

-a.2- signal quality (level of signal received) is to be seen as well at inverter AND at system point (communication is peer <-> peer)
by using SunnyExplorer then you will see two values -> the signalquality as received by inverter -> the signalquality as received by "system"
sbfspot does inform on the quality as seen by inverter

-a.3- "experienced" = distance after elemination of negative influences:
example -> obstacles (walls / . . . ) have a negative impact

-a.4- if you apply a class2 BTdongle (effective distance <<10meter) you will find yourself very often in the situation that:
  • you can use a direct wire (RJ485 / speedwire / . . . ) instead of the BT connection
-a.5- if you were lucky to catch a quality class1 BTdongle (reality distance <<40meter) be happy

rule of thumb for distance: the closer the better

a simple work around for the distance
-> apply an USB extension cable (~5meter) to move the dongle closer to inverter


-b- stability of the "firmware" AND "hardware" of the BTdongle
background information: i own three dongles
-B.1- -> first type - class1 - does never fail (more then 30days) without a problem
-B.2- -> second type - class2 - after xx-hours (xx between 4<->90) the dongle gets screwed-up (needing physical dis->reconnect to reset the status
-B.3- -> third (part of my system) - class2 - aftrer xx-hours - gets instable

rule of thumb: if you experience unstable interactions -> try to find a better one ->

kr wim
Aug 5, 2015 at 7:36 PM
Edited Aug 5, 2015 at 7:37 PM
Hi Joe,

I applied a CSL Bluethoot dongle, with raspberry pi installed one story below the inverter and till now works fine. I did notice in your pvoutput graphs two things:
  1. the voltage vary so much, from 240 to 286 V just in few hours: also for this reason I would prefer a device with measure W (accounting for voltage changes and cosfi) rather than A
  2. from daily profile I see that the power used follows very well the power produced (remaining just below it): that's great (exactly what I want)!. Have you put on somehow a system which absorb the excess of power sold to network company?
    thanks and happy son's marriage
Kr Wim thanks for the explanation about the 12 W. Now it's clear the origin of that.
Aug 5, 2015 at 8:19 PM
dear joe, -> from your info you have a fluke's clamp great stuff
  • depending on the type of clamp (see spec's by fluke) it might measure DC and AC currents !!!
  • take the time to read the specifications
    I do use a Fluke as well but it is specified -> as "minimum" measuring unit 50mA's this is really 50mA
    the range is 50mA -> 500A so my clamp is not fit for low power environments
  • it is one of the drawbacks of current-cost tools, that you can only measure ACcurrents,
kind regards wim
Aug 5, 2015 at 9:35 PM
Hi Guys,

Wim - I have to admit that my original bluetooth dongle came from eBay and cost £3 (UK pounds) - as we say, you get what you pay for! My new one is still not great, but it is the only one sold by our local computer shop so I had no choice. I have moved the Pi so it is just about 800mm away from the inverter, so distance is not an issue now. The good news is that it has given no trouble so far, but I will need a couple more days to feel confident.

Gabriele - you are correct in both your observations! The voltage shown here is pretty meaningless, it is the DC voltage generated by my East-facing string of 8 panels. Since there are also 12 West-facing panels in a separate string, don't try to find very much meaning in those figures! Yes, I do have a device to divert surplus PV power to the water heater, so you will often see the consumption tracking the generation. The device I have is the "Solar iBoost" and I have been very pleased with it. It is capturing about 100kWh per month in the summer, which would otherwise have been exported to the grid.

I am not very pleased with the weather at the moment - can I blame it on bluetooth?

Regards,

Joe.
Aug 5, 2015 at 10:13 PM
This is somewhat off topic, but I would really like to understand the aversion to feeding the grid in Europe and the UK. Most states in the US. offer net metering. At the risk of stating the obvious, the meter runs forwards (consumption) or backwards (generation).
If at the end of the month there is a net positive balance, it is carried over to the next month. At the end of the billing year, if there is a net positive, we receive a payment for the for the net positive amount. That amount is based on the billing years
average KWHr rate (mine is $0.12 US / KWhr in the current year). Now, let me say, I have never run a net positive in any month. Our house has two heat pumps so we use more electricity than homes with gas or oil heat. So, what's the rub with the UK/EU grid
feed? 73 Ron / W4MMP On 8/5/2015 17:35, penman wrote:
Aug 5, 2015 at 11:01 PM
Ron,
I can only speak for the UK, but our meters only run forwards, unless you are very lucky and have a very old one, which unfortunately they will soon change for a new one! We receive a small payment for export which is calculated as 50% of your total generation. So, you get the 50% even if you export nothing. So, it becomes a "point of honour" to export as little as possible! For early adopters of PV panels, there is a very generous payment for each kWh generated, irrespective of whether you use it yourself or export it. This is in addition to the 50%export tarrif. Sadly the present Feed in Tarrif is much less generous, though folks like me who qualified for the early generous tarrif keep it for 25 years. It was just about the best investment I ever made!

Regards,
Joe.
Aug 5, 2015 at 11:57 PM
Hi Joe,
OK, thanks for the explanation. This was just one of things that I had in the back of my mind when it comes to solar. Strange, in the states when solar is installed, the local power company (almost) automatically installs a net metering (backwards/forwards) meter. The payment rate is the same as the consumption rate. So far the states have been pretty fair when it comes to the treatment of solar generation. However, my state (Virginia) just gave notice that the size of new residential systems may not produce more energy than the average yearly consumption. Seems the power companies are now starting to loose profits due to the up tick in solar. Go figure :-(

Thanks again,
73
Ron / W4MMP
On 8/5/2015 19:01, penman wrote:

From: penman

Ron,
I can only speak for the UK, but our meters only run forwards, unless you are very lucky and have a very old one, which unfortunately they will soon change for a new one! We receive a small payment for export which is calculated as 50% of your total generation. So, you get the 50% even if you export nothing. So, it becomes a "point of honour" to export as little as possible! For early adopters of PV panels, there is a very generous payment for each kWh generated, irrespective of whether you use it yourself or export it. This is in addition to the 50%export tarrif. Sadly the present Feed in Tarrif is much less generous, though folks like me who qualified for the early generous tarrif keep it for 25 years. It was just about the best investment I ever made!

Regards,
Joe.

Aug 6, 2015 at 8:31 AM
Hi Ron,

Back in 2011 when I installed my PV system, the UK government was very keen to increase the solar take-up because it helped their sustainable energy targets. To get the industry moving they introduced a "Feed-in Tariff" of £0.43 per kWh generated, but they also capped the size of domestic installations at 4.0kWp - so it wasn't just a formula for generating money! In addition there was an export tariff of £0.031 per kWh exported. If an export meter was fitted, that reading would be used to calculate the export, otherwise the 50% assumption would be used. If you take no action to shift your consumption to the times when the sun shines, the 50% is uncannily accurate for most domestic situations! Both these tariffs are linked to the "Retail Price Index" and are updated yearly - currently standing at £0.488 and £0.034. By the way, the retail price of electricity is around £0.12 per kWh

Now that pvccupload seems to be behaving well, I have a couple of questions for you. Firstly, instead of reporting Udc1 to PVO, is it possible to use Uac1? I can't see that option in the configuration files, but I could easily have misunderstood something! Secondly, SBFspot creates a year directory, currently 2015, and that's where the .csv files are placed for pvccupload to pick up. Presumably SBFspot will create a new directory, 2016 on 1st January and I will have to re-configure pvccupload to look there instead. Is that correct? I see I can configure SBFspot so that it doesn't create the year folder - is that a better option?

Finally, is the CC Meter's clock used at all? I am wondering what happens at DST time changes. On the Owl meter, the clock is synchronised to the clock of the PC it's connected to, but I don't think the CC clock has that facility. I am just trying to anticipate possible future problems, so I can be ready with a solution!

By the way, the CC consumption reading was about 5% below the Owl yesterday. I'll continue to monitor it for a little while, but I think I will need to establish which is the more accurate before I actually do anything. I turned my workshop upside down yesterday looking for my Fluke current clamp. I'm sure it was in a black and yellow pouch, but I can't find it at the moment....

Kind regards,

Joe.
Aug 6, 2015 at 11:28 AM
Hi Ron,
similar situation in Italy; when I set-up the PV plant in early 2011 I got a fixed prize of 0,47 € for each kWh produced (indipendently from the consumption).
I do not want to sell energy to the network because there is no kWh balance between energy inserted and energy taken from the network (due to taxes and something else):
  • when I release energy into the network they recognize me 0,04 €/kWh (in addition to fee for produced energy). Even less cause I shall pay for the meter and similar services given by network supplier
  • when I take energy from the network I pay from 0,12 to 0,16 €/kWh (depending on the week day and hour)
    So, I should find the way to use at best the energy produced during the day instead of giving for "almost free" to the network. Of course the best solution would be the electrical accumulation...waiting for Tesla batteries
ciao,
gabriele
Aug 6, 2015 at 1:33 PM
Hi Jo, NOTE to SBF: I hope you do not mind us high jacking your forum to discuss issues not 100% related to SBFspot. If you would like us to take this offline, no problem. Just let us know. 73 Ron / W4MMP On 8/6/2015 04:31, penman wrote:
Aug 6, 2015 at 1:42 PM
Joe,
One more thing: Here is my cronjob entry for SBFspot:
* * * * * /home/ron/sq1/smaspot -ae0 -am0 -d5 -v5 -cfg/home/ron/sq1/sq1.cfg > /home/ron/sq1/logs/smaspot.log 2>> /home/ron/sq1/logs/smaspot.err ;sync;sync;sync
(I rename the SBFspot executable to smaspot as all my scripts are based on the old name)

Obviously make the necessary changes to match your configuration. Notice that SBFspot runs every minute. This helps avoid issues with the blue booth connection flaking out from time to time. It also provides a better assessment of Udc1 as pvccupload averages the one minute values over a 5 minute period)
73
Ron / W4MMP
On 8/6/2015 04:31, penman wrote:

From: penman

Hi Ron,

Back in 2011 when I installed my PV system, the UK government was very keen to increase the solar take-up because it helped their sustainable energy targets. To get the industry moving they introduced a "Feed-in Tariff" of £0.43 per kWh generated, but they also capped the size of domestic installations at 4.0kWp - so it wasn't just a formula for generating money! In addition there was an export tariff of £0.031 per kWh exported. If an export meter was fitted, that reading would be used to calculate the export, otherwise the 50% assumption would be used. If you take no action to shift your consumption to the times when the sun shines, the 50% is uncannily accurate for most domestic situations! Both these tariffs are linked to the "Retail Price Index" and are updated yearly - currently standing at £0.488 and £0.034. By the way, the retail price of electricity is around £0.12 per kWh

Now that pvccupload seems to be behaving well, I have a couple of questions for you. Firstly, instead of reporting Udc1 to PVO, is it possible to use Uac1? I can't see that option in the configuration files, but I could easily have misunderstood something! Secondly, SBFspot creates a year directory, currently 2015, and that's where the .csv files are placed for pvccupload to pick up. Presumably SBFspot will create a new directory, 2016 on 1st January and I will have to re-configure pvccupload to look there instead. Is that correct? I see I can configure SBFspot so that it doesn't create the year folder - is that a better option?

Finally, is the CC Meter's clock used at all? I am wondering what happens at DST time changes. On the Owl meter, the clock is synchronised to the clock of the PC it's connected to, but I don't think the CC clock has that facility. I am just trying to anticipate possible future problems, so I can be ready with a solution!

By the way, the CC consumption reading was about 5% below the Owl yesterday. I'll continue to monitor it for a little while, but I think I will need to establish which is the more accurate before I actually do anything. I turned my workshop upside down yesterday looking for my Fluke current clamp. I'm sure it was in a black and yellow pouch, but I can't find it at the moment....

Kind regards,

Joe.

Aug 7, 2015 at 10:27 AM
dear jo, ron, danielle . . . for the point of the further evolution of consumption-data handling within sbfspot

I have raised c123 - proposing to sbf being the site-manager - asking to create

a seperate block "applications" (or extensions) should be usefull to host
information about parallel developments - applications that depend on sbfspot as a main source of information

the new block should work like "discussions" -
the seperate block allows to keep actual block "discussions" reserved to "sbfspot"

the "application" block will be the breeding room till
sbfspot users like "joe" . . . . found out how to progress with that "application"


that new block can include the issues:
■C095 which is about CSV-file-importer
■C044 which is about SMA's energy meter
■the discussion items about "consumption"
■wishlist item for the same demand

from the discussion i believe that there is a common understanding that
handling the "consumption" data from . . . . is NOT YET seen as an extension to sbfspot

that position might change if "SMA's energy meter" or other "SMAproducts" gets integrated in sbfspot
Aug 10, 2015 at 1:24 PM
question about the setup of current-cost with windows8

on cc-website is indicated that USB-data cable does not work with windows8

has any one experience on this subject,

thank you, wim
Aug 10, 2015 at 5:09 PM
Hi Wim,
This may not be the same, but I think the CC monitor USB cable is actually a USB to serial (com port) device. I use one of these for programming electric wheelchair controllers and I have never got it to work on Win8. Fine on Win 7 and XP. On Win 8, the driver loads OK, the virtual com port appears to be there, but the software can't see it. I assumed this was just a driver problem, but maybe there's more to it? I didn't pursue it, easier to use an old laptop with XP!

Regards,
Joe.
Aug 10, 2015 at 6:57 PM
Hi Joe,

Ok, attached is a new version of pvccupload. It will send any single value that is recorded in the xxxx-Spot-{data}.csv file to pvoutput.org. I made the modification generic so any value can be sent. Some values do not make sense to be sent to pvoutput.org but this was a quick change with no real error checked as to the validity of sending any given field. The next release will add validity checks so that fields such as date/time, device name and such are kicked out as errors when pvccupload starts.

Currently your ini file should look something like this at the bottom of the file:
Y; Move data files to archive
placeholder;

Change it to this:
Y; Move data files to archive
16;
placeholder;

This should cause pvccupload pick up the the Uac1 value. Install the new version of pvccupload and restart it. If you would like to restart pvccupload to replay data simply change the restart date in the ini file to replay older data. Give it a go and let me know if it works. I did a quick test to my pvoutput.org system and it did pick up the Uac1 values.

Below are the available SBF-spot fields:

0 = dd.MM.yyyy HH:mm:ss;
1 = DeviceName;
2 = DeviceType;
3 = Serial;
4 = Pdc1;
5 = Pdc2;
6= Idc1;
7 = Idc2;
8 = Udc1;
9 = Udc2;
10 = Pac1;
11 = Pac2;
12 = Pac3;
13 = Iac1;
14 = Iac2;
15 = Iac3;
16 = Uac1;
17 = Uac2;
18 = Uac3;
19 = PdcTot;
20 = PacTot;
21 = Efficiency;
22 = EToday;
23 = ETotal;
24 = Frequency;
25 = OperatingTime;
26 = FeedInTime;
27 = BT_Signal;
28 = Condition;
29 = GridRelay;
30 = Temperature

73
Ron / W4MMP
On 8/10/2015 13:10, penman wrote:

From: penman

Hi Wim,
This may not be the same, but I think the CC monitor USB cable is actually a USB to serial (com port) device. I use one of these for programming electric wheelchair controllers and I have never got it to work on Win8. Fine on Win 7 and XP. On Win 8, the driver loads OK, the virtual com port appears to be there, but the software can't see it. I assumed this was just a driver problem, but maybe there's more to it? I didn't pursue it, easier to use an old laptop with XP!

Regards,
Joe.

Aug 10, 2015 at 7:25 PM
Hi Guys,
The issue has been (for a long time) with the bogus (read counterfeit) Prolific converter chips used in many USB serial converter devices. So, Prolific took the draconian measure of creating drivers that will only work with genuine Prolific chips which disabled many many users devices when the user updated the driver (usually via Window Update). Companies such as Current Cost have kept the old drivers in their repositories so that when customers purchase their equipment (the USB/serial interface cable(s)), the devices work. The fact that the companies are selling devices with a counterfeit device is not their fault. They had no way of knowing this ahead of time. The issue now is with newer releases of Windows and the old drivers failing. I'm not sure what the solution is. It a bit of a catch 22 situation.

BTW: FTDI (another manufacture of USB serial converters) attempted the same draconian measure of producing a driver that would work with only genuine FTDI chips. But this time the back lash from the user community was so severe, they quickly restored the previous drivers in the repositories.

One solutions is to move to Linux :-) . No issues there as the drivers are not supplied by the vendor and are open source. So if something goes wonky, there is a whole gaggle of folks ready to fix it.
73
Ron / W4MMP
On 8/10/2015 13:10, penman wrote:

From: penman

Hi Wim,
This may not be the same, but I think the CC monitor USB cable is actually a USB to serial (com port) device. I use one of these for programming electric wheelchair controllers and I have never got it to work on Win8. Fine on Win 7 and XP. On Win 8, the driver loads OK, the virtual com port appears to be there, but the software can't see it. I assumed this was just a driver problem, but maybe there's more to it? I didn't pursue it, easier to use an old laptop with XP!

Regards,
Joe.

Coordinator
Aug 11, 2015 at 7:24 PM
rlpatton wrote:
Hi Jo, NOTE to SBF: I hope you do not mind us high jacking your forum to discuss issues not 100% related to SBFspot. If you would like us to take this offline, no problem. Just let us know. 73 Ron / W4MMP On 8/6/2015 04:31, penman wrote:
No problem at all Ron
Aug 12, 2015 at 8:14 AM
Edited Aug 12, 2015 at 11:33 PM
to penman | joe about the usb-serial cable for cc equipment - this cable has "active components"
  • the supplier (Prolific) of the (curCost) cable ships new cables with another chipset,
    this new cable and its drivers should support win8 and win10 (sic)
  • Prolific states that cables with the old chipset will not work on win8 ->
your right on the point - :
it consumes less time to continue working with an "old" XP (or vista) PC
till the "market" has created sufficient tools to work around this type of "limits"

kr wim
Aug 12, 2015 at 11:37 PM
dear ron,

can you inform on the "host" website / environment with the new version of pvccupload

kr wim
Aug 13, 2015 at 7:05 PM
Hi WIM,
Sure I will take care of that in a few days.
73
Ron / W4MMP
On 8/13/2015 13:50, SillieWimons wrote:

From: SillieWimons

dear ron,

can you inform on the "host" website / environment with the new version of pvccupload

kr wim

Aug 19, 2015 at 8:41 AM
Hi all,

thanks for this thread.. I started a thread about upload of consumption data, however I was not aware of this vivid discussion here. I do not want to take over this thread with my issues. I just have a few simple questions after reading through all of the replies here.
  1. I was not aware of pvccupload as another possibility to combine consumption with SBFspot values to upload to PVOutput. Form a first glance it seems that it is tailored for a specific meter called CC. I do not have such a meter. I have a meter which sends his data via SML over a D0 optical interface. I currently read the results via an USB-IR Dongle and covert the data in the SML protocoll to a meaningful value like kwh and wh. Could pvccupload also work with my setup, if I manage to somehow store the readed values in a CSV file ?
  2. How is pvccupload installed ? Just untar and that's it ? Do i have to compile the sources somehow ? I'm not a linux pro, that's probaby my biggest issue right know. So I do not know how to really configure pvccupload. Jon mentioned to maybe put up a guide with his experiences. That guide woud be awesome to have !!
Thanks,
Axel
Aug 19, 2015 at 2:50 PM
hej axel,

-a- the reply to the first question is simple !!! thanks to ron
pvccupload started as a tool to "merge" (read and handle) information from different sources,
one of his original sources was "CurentCost" measuring equipment (output in .csv format)
-> inputs:
SBFspot is one of the "data" sources that can be used,
"CurrentCost" (equipment and sw) is one of the measuring tools supported,
basic input format is xxx.csv files
-> outputs:
.csv files and/or upload to PVoutput (or . . . )

-b- for the second question
-b.1- ron has done a great job on documentation about the way his tool operates,
unfortunate "google" stopped its support as a "development-environment"
you'll get the available info when searching on the web: -> google code or sourceforge or . . (documents and info still available in "read-only")
-b.2- the next step is open for support by: "joe" or "axel" or . . .

-c- projects like sbfspot and pvccupload take time and support to deliver commonly usefull
information eilas patience often needed

kind regards wim
Aug 21, 2015 at 7:05 PM
Hi,

The pvccupload application suite only supports Current Cost meters and SBFspot (and by extension, SMA inverters). Sorry, but I have no intention to support other power meters or inverters. I will entertain the notion of making changes to the current suite for enhancements but still only for CC meters and SBFspot. The reason I chose CC meter(s) was at the time it was the only reasonably priced meter in the U.S. market and SMA for similar reasons.

73
Ron / W4MMP
On 8/21/2015 14:53, SillieWimons wrote:

From: SillieWimons

hej axel,

-a- the reply to the first question is simple !!! thanks to ron
pvccupload started as a tool to "merge" (read and handle) information from different sources,
one of his original sources was "CurentCost" measuring equipment (output in .csv format)
-> inputs:
SBFspot is one of the "data" sources that can be used,
"CurrentCost" (equipment and sw) is one of the measuring tools supported,
basic input format is xxx.csv files
-> outputs:
.csv files and/or upload to PVoutput (or . . . )

-b- for the second question
-b.1- ron has done a great job on documentation about the way his tool operates,
unfortunate "google" stopped its support as a "development-environment"
you'll get the available info when searching on the web: -> google code or sourceforge or . . (documents and info still available in "read-only")
-b.2- the next step is open for support by: "joe" or "axel" or . . .

-c- projects like sbfspot and pvccupload take time and support to deliver commonly usefull
information eilas patience often needed

kind regards wim