Does anybody use raw data storage on Local Blynk Server?

@Costas My thoughts are, that csv’s are exactly those files planned to be removed. History data for history graphs are supposedly stored in history*.bin files (there are for minute, hour, week, etc time span), but then again - data kept in csv’s need to be kept somewhere, as exporting seems to be separated from those history*bin files. I just don’t like CSV as a storage bin, as they are huge after time, but that is my story… @Gianca You need to wait for @Dmitriy response.

I accept it’s not entirely clear but my understanding is this:

There was a storage system on the server before csv was made available and that the proposal is to remove the old storage system.

Is this correct @Dmitriy ?

Your reference to History graph does confuse me though because I believe data is currently available that is not tied to a History graph. Maybe this is what you mean by " raw data". So unless pins are tied to a history graph would this data no longer be available?

@Costas @Dmitriy Just to give other information, my final user are “institutions” that cannot deal only with a telephone. They need to receive graphic data thru an official system. For this reason I cannot use history widget.

Exactly. Raw data is csv files. History csv is composed from “history_*.bin” files and this is seprate thing from raw CSVs.

@Gianca what exactly CSV files do you use from above :)?

@Dmitriy so if the raw data (csv) was removed then history csv would still be available from the history_*.bin files.

But other than using the email csv feature in the graph widget is there any way to access the bin file data in csv format?

Correct.

Via HTTP API.

@Dmitriy This is one of the file that i use (just an example): 199382847-0_v81.csv

Thanks

@Gianca http://[server_url_or_ip]/[token]/[pin] downloads a csv file in the form name@domain.com_projectnumber_pin.csv.gz

Would this work for you?

Presumably v81 is the pin reference but what is the rest and where do you currently get this data from?

@Costas

No. The software that I developed is tailored on the actual system in use

@Costas

@Costas

199382847-0_v81.csv is exactly the name of the file that I use to pick up data and create graph thru AM Chart. Of course v81 is the virtual pin.

@Costas

I get this file from the data/user_name folder of my local server.

Anyway I’ve spent more than 1 year of hard work to develop my system. It would be really very frustrating if I should change it and reinvent a new one !!!

Personally I think having raw data available on the server could be valuable thing for certain projects, but if the majority of users don’t need it, maybe have it off by default?

I fully understand.

My understanding is that the csv files on the server are considerably larger than the bin files and when you have 20,000+ devices, and growing, I think you will understand why Blynk are looking to make changes.

I don’t pretend to know how the bin to csv conversion works but perhaps @Dmitriy could provide the basic details and through server.properties perhaps opt in to csv files being created automatically.

@Jamin 4GB?

I don’t see the real problem. The dimension of files on a server is a problem of the user and not of Blynk. Let’s decide to a user if he needs CSV files or not. I vote to leave things unchanged by default and if a user don’t want CSV files he will modify the properties file.
Anyway I build up my system using the features that Blynk offers. That’s all. Now, if these features change and a developer is obliged to rework all projects already delivered, I believe that it’s not a positive point.

@Dmitriy: by the way, can you think about extending possibility to export data through API http://ip:port/auth_token/data/pin with time frame [BEGIN=, END=] (be it native UNIX in millis adopted by Blynk, or a “human readable” data - whatever). I read about this idea somewhere here, and think it would be very helpful in cases such as @Gianca’s, and supposedly easier to implement for less experienced users (such as me :wink: )

@Gianca sorry, already removed in 0.22.0 release. We have alternative approach for raw data - DB. So now you have 2 options : turn off updates and leave things as it is, add DB to your local server. I understand your pain, but old code requires support and all this is free and we can’t afford that.

We are doing Web for that. Users don’t need to know anything about “How data is stored”.

The biggest majority of Blynkers are using Blynk’s server not a local server.

@Costas we don’t use csv on cloud. We have DB there.