Webhook and reading web data

Hi all,
I have read through a topic from last year relating to reading and parsing data imported from a website into Blynk using the webhook widget. In it, there were some comments around building some capability into Blynk to simplify, particularly the parsing of the data and extracting specific values. I’m wondering if anyone can tell me whether any of these ideas were implemented?
I am building a project that involves importing a small text file of current weather conditions from my personal weather website. From the imported data, I want to extract just the daily rainfall figure and display it in a Blynk project.
I have setup a test webhook widget and and sketch and am successfully importing the data and displaying it on the serial monitor (I’m calling the data using the Blynk timer approach).
The next bit around parsing the data and extracting the value I need has me stumped. I read through the parsing code in last year’s post and definitely beyond my capability as a non-coder. I’m hoping there is a simpler option. The text file I’m importing is below. The rainfall value I’m looking to extract is the 9th value.


28/02/19 17:06:25 22.7 78 18.7 13 16 360 0.0 0.0 1005.7 N 3 km/h C mb mm 93.6 +0.1 3.0 17.4 0.0 27.7 49 22.7 -0.5 28.7 09:22 22.4 16:49 22 14:18 40 16:00 1006.5 10:12 1005.2 14:40 3.0.0 3046 24 22.7 29.1 0.0 0.00 0 279 0.0 3 1 0 W 1690 ft 23.2 0.0 280 0 

Cheers
Steve

This isn’t a Blynk related question really, but I’ll be kind with you :angel:

Your weather station data uses spaces as separators between data fields and the exact position of the rainfall data within the string can vary if we get larger or smaller numbers for some of the preceding data fields. For example, if the 4 data item increased from 78 to 105, it would push the rainfall data along by one character (I guess you already realised this).

Therefore, the only way to locate your rainfall data within the string is to start at the beginning and search for spaces. When we find the position of the 8th and 9th spaces then we know that the data we want is located between those two positions within the string ans we can then pull it out.

The code below is much more verbose than it needs to be, but that makes it easier to follow (and to debug). Hopefully you’ll understand the principal and be able to modify it to pull out other data if needed.

//                             1         2         3         4         5         6         7         8         9
//                    0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789
String WebhookData = "28/02/19 17:06:25 22.7 78 18.7 13 16 360 0.0 0.0 1005.7 N 3 km/h C mb mm 93.6 +0.1 3.0 17.4 0.0 27.7 49 22.7 -0.5 28.7 09:22 22.4 16:49 22 14:18 40 16:00 1006.5 10:12 1005.2 14:40 3.0.0 3046 24 22.7 29.1 0.0 0.00 0 279 0.0 3 1 0 W 1690 ft 23.2 0.0 280 0";
// Count the spaces           1        2    3  4    5  6  7   8   9   10
// We want this data                                           xxx

// In this example, the characters we want are in positions 41, 42 & 43 (between the 8th and 9th spaces)
// Note that C++ uses zero indexed strings, so the first character is position 0

 
int Position_of_Space_8;  // when we find out where the 8th space is, we'll save that info here
int Position_of_Space_9;  // ditto for the 9th space

String Rainfall_Text;     // we'll extract the rainfall data (as text) and store it here
float Rainfall_Number;    // we'll then turn the text into a number so that you can do some maths on it if you want to



void setup()
{
  Serial.begin(74880);
  Get_Rainfall();
}


void Get_Rainfall()
{

  int Starting_Point = 0; // This is where in the WebhookData string we start searching for spaces

  for (int i=1; i<=9; i++)  // We'll loop through and find the first 9 spaces in the WebhookData string
  {
    int Current_Position = WebhookData.indexOf(" ",Starting_Point);  // Find a space, looking from our Starting_Point
    
    Serial.print("Position of space "); // Print out the position of the first 9 spaces...
    Serial.print(i);    
    Serial.print(" is ");
    Serial.println(Current_Position);
       
    Starting_Point=(Current_Position+1); // Next time we go around the loop and search, we want to find the next space, so start searching at the position of the last space + 1


    if (i==8) // If we've found the 8th space then make a not of it's position
    {
      Position_of_Space_8 = Current_Position;
    }

    if (i==9) // If we've found the 9th space then make a not of it's position
    {
      Position_of_Space_9 = Current_Position;
    }
  } // repeat the for loop



  // We get to this point when the for loop has completed

  // Here we grab the text that's between space 8 and space 9
  // We don't want the leading space, so we start at Position_of_Space_8 + 1 ...
  Rainfall_Text =  WebhookData.substring((Position_of_Space_8 +1),Position_of_Space_9); 


  Serial.print("Yay! we found the rainfall text, it's ");
  Serial.println(Rainfall_Text);

  Rainfall_Number = Rainfall_Text.toFloat();

  Serial.print("Rainfall as a number is ");
  Serial.println(Rainfall_Number);

  Serial.print("To prove it works, we'll add 0.3 on to our rainfall value. The result is ");
  Serial.println(Rainfall_Number + 0.3);  
}



void loop()
{

}

It produces the following output in the serial monitor:

Position of space 1 is 8
Position of space 2 is 17
Position of space 3 is 22
Position of space 4 is 25
Position of space 5 is 30
Position of space 6 is 33
Position of space 7 is 36
Position of space 8 is 40
Position of space 9 is 44

Yay! we found the rainfall text, it's 0.0
Rainfall as a number is 0.00
To prove it works, we'll add 0.3 on to our rainfall value. The result is 0.30

Because I’m feeling especially generous, I’ve given you text and floating point versions of the rainfall data. If you wanted, you could use the floating point version to do some maths - for example if the rainfall is greater than 5 send an alert saying that it’s raining cats & dogs, if it’s greater than 10 then send an alert saying that you should buy a boat.:rowing_man:

If you can’t follow what I’ve done then just ask.

Pete.

I don’t know you, but I’d use the strtok() function. Check this link for more info. And it also uses less memory.

Yep, probably a much better solution :grinning:

Pete.

char data[] = "28/02/19 17:06:25 22.7 78 18.7 13 16 360 0.0 0.0 1005.7 N 3 km/h C mb mm 93.6 +0.1 3.0 17.4 0.0 27.7 49 22.7 -0.5 28.7 09:22 22.4 16:49 22 14:18 40 16:00 1006.5 10:12 1005.2 14:40 3.0.0 3046 24 22.7 29.1 0.0 0.00 0 279 0.0 3 1 0 W 1690 ft 23.2 0.0 280 0";
char* p = strtok(data, " ");
for (int i = 1; p != NULL; i++) {
   Serial.printf("[%d] %s\n", i, p);
   p = strtok(NULL, " ");
}

[1] 28/02/19
[2] 17:06:25
[3] 22.7
[4] 78
[5] 18.7
[6] 13
[7] 16
[8] 360
[9] 0.0
1 Like

Hi all,
Many, many thanks for your help. I had hoped I would get a few pointers but had not expected people to take so much time to provide code. I will start playing with both solutions - great learning experience for a non-coder. Very much appreciated.
@PeteKnight Pete, I agree the coding solution is not a Blynk issue. The original thrust of my query was to see if there had been any additional development on some thoughts @Pavel provided in a similar post last year around providing some parsing capability within Blynk itself for extracting values from theses types of webhook files from a 3rd party. Doesn’t sound like it has moved.

Cheers
Steve

1 Like