Defer Writes

Ideas
1

Hi,

I just realized that data is sent as soon as the virtualwrite call is made. In my project I am seeing most of the processing time taken up by the virtualwrite() calls and blynk.run() executes almost instantly. I am not sure this is the best way of doing it.

I have lots of virtualwrites() going and its hard to estimate the execution time of several timer functions.

I thought that virtualwrite() would store the new data and blynk.run() would send it all (this would be why you should only have one blink.run())

However, the opposite is happening. Code may have lots of virtualwrites (where all the work seems to be done) and blynk.run() does nothing? If the magic happens in the write functions, what does blynk.run() actually do?

Is there a way to defer the writes such that it only happens during blynk.run()?

2

I changed your topic to a Need Help categoryā€¦ since I highly doubt Blynk is going to just drop what they have already built and redesign it in whatever unique way you thing it should be :stuck_out_tongue:

I am not a developerā€¦ but they are busy and might not respond right away, so I will try to assist with some clarifications as best as I am able.

Not sure how you could have come up with that impressionā€¦ have you done any previous Arduino coding?

Blynk.run() needs to be in the void loop() at the very leastā€¦ it is Blynkā€™s housekeeping command and basically keeps everything running. And since it is in the void loop() which already runs hundreds or even thousands of times a second, there is no need to ā€œdeferā€ commands until it runs.

As for all your Blynk.virtualWrite() commands (and all other Blynk commands), they are supposed to process immediately by the library,if not directly by Blynk.run().

If you want ā€œdefermentā€ simply code in as such; As you have already indicated that you use timers, this should not be an issue. And estimating command process timing and thus ideal timer settings is part in parcel with learning to properly code in the first place. You could even use milis() and Serial.print() commands to determine exact start and stop times during code development.

That said, sending out ā€œlotsā€ of commands may not be an issue anymore, as such appears to be now automatically regulated via the new 0.5.0 library.

Hopefully this clarifies it for youā€¦ :slight_smile:

If you are still unclear on something, please ask.

3

Thanks for your quick reply.

Sorry, but I think the topic should still be Ideas.

And it certainly not a unique way of doing things. Queues are used everywhere (every RTOS has them. :wink:)

I have used many kinds of micro controllers but I am not used to the arduino way of doing it. There is too much abstraction and I am not really sure of whats going on under the hood; and I find it difficult to optimize this way.

Everyone just says donā€™t use delays, and call blynk.run() in loop() as frequently as possible. This lead me to believe that most of the work happens in blynk.run() which was an incorrect understanding on my part, but I digressā€¦

My main point was that I am noticing about 150-200ms of delay when I am calling virtualwrite(). The way I have structured the code, there is about a dozen of calls peppered throughout the program and all of these are introducing delays (much longer than my expected timer period) which made it difficult to debug. I guess I needed to pay closer attention to the docs. Does that mean I should account for this time for each blynk call? Itā€™s not very convenientā€¦

My deferred write idea was to have all the virtualwrite() calls to write to memory (i.e. queue the data) and then when the next blynk.run() comes around, all the data is queue is sent in one go.

This can allow you to structure the data stream depending on your blynk.run() calls and would make the code more deterministic. In this case, you can call blynk.run() once or place it in higher priority places where youā€™d want more frequent or urgent updates.

It would also de-couple your code from the blynk stuff. Right now the code is very tightly coupled and execution depends on the placement of your blynk calls.

The way it is done now is not very deterministic. I donā€™t see why the blynk calls should occur immediately. If this is what is desired, call blynk.run() right afterā€¦

4

Well, it sounds like you donā€™t need/want assistanceā€¦ and I am not interested in debating the hows and whys of Blynkā€™s current process (i.e. Iā€™m not a Developer :stuck_out_tongue: )ā€¦ So I will switch it back and leave this one for the Developers to answer or ignore.

Maybe you can contribute new library code over on GitHub?

5

I would love to see an implementation of such a function, but not at the expense of the current functionality!

All in all, a:

  • virtualWrite (sends instantly)
  • virtualWriteQ (Q for queued, sends on next .run)
6

You meen the one 0.1ms from the last .runā€¦ or the one 0.1ms after that one? :smile:

(Disclaimerā€¦ 0.1ms is only an estimated recurrence time for void loop() cycles, as actual time for processing Blynk.run(); is dependent on clock speed of MCU - and bad code - and may varyā€¦ as will colour and texture due to manufacturing differencesā€¦ and contents may settle during shippingā€¦ :stuck_out_tongue_winking_eye: )

7

Well tbh not all applications use the loop :slight_smile: in my case, for battery powered sensors I have no need of loop. However the functionality of queuing up writes would ease up on some aspects of my code :stuck_out_tongue:

8

@Fettkeewl Oh, I understood you :wink: There is always a niche market for some particular way of doing somethingā€¦ I guess my point is that what I think the OP is really looking for is a totally different variation of Blynk, designed from the ground up for an RTOS, and not really meant for the average Joe using Arduinos & ESPā€™s variation of C++ & IDE and looking at basic IoT integration.

@BoogaBooga Actually, if you are seriously looking for changes like that, then I suggest you inquire over on the business side of Blynkā€¦ They can customise anything you need - https://www.blynk.io

2 Likes
9

honestly, for me, one of the greatest ā€œfeatureā€ of blynk is that super fast (realtime) speed.

when you send something - it is sent - instantly. why it should be sent with delay??? the power of hardware and mcu is the super fast speed. imagine if your keyboard or lamp switch or volume control would respond with delay:

that would be pretty annoying - or even completely unusableā€¦

blynk virtualwrites quite often are used to give feedback to the user: you do something in the hw, and expect some feedback in the app. for a great user experience, this is best to happen asap.

the idea with deferring writes it is quite simply implementable by anyone, without slowing down the basic blynk functionality:

just create a blynk.timer instance and put all blynk virtualwrites you wish to defer into the callback function. this way, you can defer all writes as much as you like (seconds, hours, days, etc) - and they will be ā€œsent in one goā€.

as for your original idea, to making ALL blynk virtualwrites any slower / delayedā€¦ give me just one reason, why it would be better from the end user pov?

1 Like
10

I guess I am used to using RTOSes where everything is made to be deterministic.

I was just suggesting an approach to make blink more deterministic; but as was said, maybe this is a need for the business side of blynk.

It is not truly implementable by us because it needs to go inside the blynk state machnine where the data can be packed,scheduled and sent correctly; one payload.

Yes you can do this on the outside (store data yourself and call a bunch of writes at one time) but youā€™d still end up doing the same amount of calls (many payloads) which will take the same amount of time. The benefit happens when you do one bigger send to the blynk server.

I would love to hear the devs thoughts on scalability. I mean, there is support dozens of Virtual pins (128 max?). How do you get all that data back in a reasonable amount of time?

If i use my metrics (i.e. 0.15ms per virtual write x 128 = 192.s) is a long time to spend transferring a few hundred bytes.

11

you can enable the serial debug in blynk, than you can study how and what happens in the background. or, for even more detail, you can take a look into the source code.

i didnā€™t study this in detail, but, as the docs says, blynk.run is for housekeeping (it maintains the connection with the server), and you can have as much blynk.run in your code as you like.

for better understanding how it works, you can compare with the serial communication protocol:

blynk.virtualwrite is like serial.println: it sends the data to the serial monitor - immediately.

i never done any scientific measurements about how long it takes 1 virtual write (and, iā€™m sure it is pretty much dependent on your local network, distance to the nearest blynk server, wifi signal, etc), but in some of my projects i use 30-40 virtual pins, without any issues / delays. the only problem iā€™ve got, only once, it was because of a bug in the old (2.3) esp core, which caused crash - after updating to 2.4, it works ok.

if indeed takes 0.15 milliseconds, as you say, for 1 virtualwrite, than for all the 128 pins it would take 19.2 millisecondsā€¦ or maybe more, i do not know.

but simultaneously updating all 128 virtual pins is quite uncommon, i see no practical reason for doing that. it is like a car, it can do 8000 rpm, but no reason for using on such high rpm.

for me, if i know that when i send data - it is indeed sent asap - that is pretty deterministic. wish, it could be so deterministic the postal service in my country :slight_smile:

12

I am not sure if the network was the problem. I use a local server on the same network and i get 10ms ping usually.

I am also not saying make blynk slower. I am asking for a mechanism that allows me to schedule the writes better. Allow for a queue that can be sent during some spare CPU cycles or send it whenever you want in the code.

Determinism has a whole other context when talking about real-time processes. In this context, sending asap is not a sufficient condition to be deterministic.

I guess whether blynk should be considered real-time is a whole other argument.

Thank you all for the discussion, and I hope we can continue it to improve blynk.

Again I am not asking to up-end blynk. If it works for you, great. I am putting forward an idea for improvements. We are all trying to squeeze the most out of our little microcontrollers and the team has created a great product towards that. There is lots more that can be optimized toward scalability and performance. Just because its good enough now, does not mean there is no room to improve.

13

If you put new batteries in your calculator the 192s will become 19.2s.

Nobody uses all 128 virtual pins and Blynkers donā€™t normally pause for 0.15ms between each write so itā€™s all done in a fraction of a second.

14

good catch.

Altho the application code does stop for 0.15ms during the blynk write. I am not sure what you mean

15

Are you using version 0.5.0 of the Blynk library?

16

Yes I am. I will post an example of what I am seeing in the next few days. Maybe I am doing something wrong.

Edit: using v0.5.0 on ESP8266 (ESP-12E)

17

I believe v0.5.0 is still officially a beta release and all previous versions operated without any delays between each virtualWrite().

For beginners to Blynk, that didnā€™t read the docs, and tried to throw everything into a messy loop(), as they did when they were using Arduinoā€™s, this created problems.

One of the biggest issues was caused by all the bugs in the Arduino ESP core that still hasnā€™t seen an official update for 18 months, although there are recent staging versions with hundreds of bug fixes.

18

I agree. I am not a fan of the arduino-isms either.
I have seen stability improvements with 0.5. I agree with the rate-limiting. I think it would be even more efficient if they can pack the data better before each send.

And another thing, the way things are now, you can send duplicate data if your value does not change. I think there should be the capability to transmit on change instead. This is a trivial thing to fix in the sketches (i.e our code), but it would be nice if the blynk library did this behind the scenes. It would also help with the flooding and reduce bandwidth (for those that are on GSM/cellular)

Sigh. Maybe i have to write a big wrapper to blynk and do my own scheduling.

19

@vshymanskyy how many ms have you set the delay at in 0.5.0?

@BoogaBooga are you using ESP in standalone mode with 2.4.0-rc2 core?

20

I am using ESP in standalone mode with ESP Arduino Core 2.3.0 release.