Okay, but now we are a little bit off topic… the post is about read data from web with Webhook…
If we would talk about Sunset and Sunrise with arduino… same as @Costas I use a library, in details I use a fork of TimeLord that is from André Gonçalves .
@naamah75 not off topic at all. Your OP asked about something that is not currently available from Blynk.
@Lichtsignaal said try DIY. You asked how and he provided the details.
I was merely pointing out that there are existing libraries available for many Webhooks rather than re-inventing the wheel.
Well, that it is, but it’s also an example on how to retrieve data from a web url.
@Costas the graph represents the sunset/sunrise times, but not the twilight times. Twilight happens in advance of sunrise and after sunset. And those are the ones which takes longer if you are further from the equator. And yes, the times are UTC, I convert them myself in another piece of code later on 
Furthermore, the API gives back different types of sunrise/sunset times whereas the TimeLord library only uses one algorithm, so each has it’s advantages/disadvantages
Only thing I can think of is that they messed up their timezones somewhere or maybe the webpage converts for local time?
I have no idea tbh. xD
@Lichtsignaal I liked your extract as it provides easy parsing of API calls without the normal additional json libraries. I was having trouble with cookies from wunderground.com which meant the index wasn’t constant.
Eventually got it running and the full sketch to parse sunrise and sunset times, without disconnecting from the Blynk server, is provided below.
/* Webhooksv2a.ino based on sketch extract by @Lichtsignaal
* http://community.blynk.cc/t/read-data-from-the-web-with-webhook/8334/6
* Terminal on V0 and button in PUSH mode on V2
*/
//#define BLYNK_DEBUG // enable for debugging Blynk problems
#define BLYNK_PRINT Serial // Comment this out to disable prints and save space
#include <ArduinoOTA.h>
#include <ESP8266WiFi.h>
#include <BlynkSimpleEsp8266.h>
#include <SimpleTimer.h> // Essential for almost all sketches
SimpleTimer timer;
WidgetTerminal terminalW(V0);
char server[] = "api.wunderground.com"; // same as: const char* server = "api.wunderground.com";
WiFiClient client;
#define debug 0 // reduced Serial Monitor output
bool validData = false; // assume bad data until validated
unsigned int sunriseseconds;
unsigned int sunsetseconds;
String sunrise;
String sunset;
int foundsunrise;
int foundsunset;
unsigned int numberofattempts = 3; // set maximum number of attempts to try to get valid data from webservice
// will try 4 times from V2 but 1 attempt is ok now
char OTAhost[] = "Webhook"; // Optional, but very useful
char ssidstr[] = "OfficeGargoyle"; // enter your Router SSID
char passstr[] = "1234567890"; // enter your Router AP password
char authstr[] = "AB012345678901234567890123456789"; // enter your Blynk token
char serverstr[] = "blynk-cloud.com"; // change to IP for local server connection
char WUNDERGROUND_REQ[] =
"GET /api/API_KEY/astronomy/q/COUNTRY/CITY.json HTTP/1.1\r\n" // enter your WUNDERGROUD API_KEY, COUNTRY and CITY
"User-Agent: ESP8266/0.1\r\n"
"Accept: */*\r\n"
"Host:api.wunderground.com\r\n"
"Connection: close\r\n"
"\r\n";
BLYNK_WRITE(V2){
int getWUsunrise = param.asInt();
terminalW.println();
if(getWUsunrise == 1){
int datachecks = 0;
while(validData == false){
getrawdata(); // keep checking for valid data
if(validData == false){ // rechecked after calling getrawdata();
terminalW.println("Invalid data, will try again");
terminalW.flush();
}
datachecks++;
if(datachecks > numberofattempts){
Serial.println("Check API");
terminalW.println("Check API");
terminalW.flush();
break; // tried enough times so bail out of while loop
}
}
gotValidData();
}
}
void gotValidData(){
if(validData == true){
terminalW.println("Data check passed");
terminalW.print("Sunrise seconds ");
terminalW.println(sunriseseconds);
terminalW.print("Sunset seconds ");
terminalW.println(sunsetseconds);
terminalW.flush();
validData = false; // resetting flag for further use
}
}
void reconnectBlynk() { // reconnect to server if we get disconnected
if (!Blynk.connected()) {
if(Blynk.connect()) {
BLYNK_LOG("Reconnected");
terminalW.print("Reconnected with local IP of ");
terminalW.println(WiFi.localIP());
terminalW.flush();
} else {
BLYNK_LOG("Not reconnected");
}
}
}
void getrawdata()
{
if(debug) { Serial.println("Disconnecting Blynk"); }
//Blynk.disconnect(); // Disconnect Blynk, cause API call has to be made
//delay(750);
int i=0;
String contents[65]; // maximum number of lines, not characters per line
String inputString = "";
foundsunrise = 0;
foundsunset = 0;
inputString.reserve(1400); // actual data approx 662 plus headers say 438 (measured size is around 1071 characters BEFORE trim
int countcharacters = 0;
if(debug) {
Serial.print("connecting to ");
Serial.println(server);
}
WiFiClient client; // Use WiFiClient class to create TCP connections
const int httpPort = 80;
if (!client.connect(server, httpPort)) {
if(debug) { Serial.println("connection failed"); }
return;
}
String url = "/api/527c5df849ee9364/astronomy/q/Cyprus/Paralimni.json HTTP/1.1\r\n"; // We now create a URI for the request
if(debug) {
Serial.print("Requesting URL: ");
Serial.println(url);
}
client.print(String("GET ") + url + " HTTP/1.1\r\n" + // This will send the request to the server
"Host: " + server + "\r\n" +
"Connection: close\r\n\r\n");
unsigned long timeout = millis();
while (client.available() == 0) {
if (millis() - timeout > 5000) {
if(debug) { Serial.println(">>> Client Timeout !"); }
client.stop();
return;
}
}
while(client.available()){ // Read all the lines of the reply from server and print them to Serial
char c = client.read();
inputString += c;
countcharacters++;
if (c == '\n')
{
inputString.trim(); // Remove dumbass whitespaces and too much CR, LF etc.
//if(inputString.startsWith("Set-Cookie:") == false){ // skip any lines that refer to setting cookies as they are once / session
if(debug) {
Serial.print(i);
Serial.print(": Received: ");
Serial.println(inputString);
}
contents[i] = inputString;
i++;
yield();
if(foundsunrise == 0){ // ensures we only search for 1st instance
if(inputString.substring(1, 8) == "sunrise"){ // sunrise
foundsunrise = i;
if(debug) {
Serial.print("Found sunrise at index ");
Serial.println(foundsunrise);
}
}
}
if(foundsunset == 0){ // ensures we only search for 1st instance
if(inputString.substring(1, 7) == "sunset"){ // sunset
foundsunset = i;
if(debug) {
Serial.print("Found sunset at index ");
Serial.println(foundsunset);
}
}
}
inputString = "";
}
}
if(debug) {
Serial.println();
Serial.println("closing connection");
}
if((foundsunrise != 0) && (foundsunset != 0)){
validData = true;
if(debug) {
Serial.println("Data is valid");
Serial.println(contents[foundsunrise]);
Serial.println(contents[foundsunrise + 1]);
Serial.println(contents[foundsunset ]);
Serial.println(contents[foundsunset + 1]);
}
int risehourlength = (contents[foundsunrise].length()); // length 11 means before 10am i.e. single digit number
int riseminutelength = (contents[foundsunrise + 1].length()); // length 12 means less than 10 minutes passed the hour i.e. single digit number
int sethourlength = (contents[foundsunset].length()); // length 11 means before 10am i.e. single digit number
int setminutelength = (contents[foundsunset + 1].length()); // length 12 means less than 10 minutes passed the hour i.e. single digit number
if(debug) { Serial.print("Sunrise at "); }
if(risehourlength == 11){
if(debug) {
Serial.print("0");
Serial.print(contents[foundsunrise].charAt(8));
}
sunriseseconds = ((contents[foundsunrise].charAt(8) - '0') * 3600);
}
else{
if(debug) {
Serial.print(contents[foundsunrise].charAt(8));
Serial.print(contents[foundsunrise].charAt(9));
}
sunriseseconds = ((contents[foundsunrise].charAt(8) - '0') * 10 * 3600) + ((contents[33].charAt(9) - '0') * 3600);
}
if(debug) { Serial.print(":"); }
if(riseminutelength == 12){
if(debug) {
Serial.print("0");
Serial.println(contents[foundsunrise + 1].charAt(10));
}
sunriseseconds = sunriseseconds + ((contents[foundsunrise + 1].charAt(10) - '0') * 60);
}
else{
if(debug) {
Serial.print(contents[foundsunrise + 1].charAt(10));
Serial.println(contents[foundsunrise + 1].charAt(11));
}
sunriseseconds = sunriseseconds + ((contents[foundsunrise + 1].charAt(10) - '0') * 10 * 60) + ((contents[foundsunrise + 1].charAt(11) - '0') * 60);
}
if(debug) {
Serial.print("Seconds from midnight to SUNRISE: ");
Serial.println(sunriseseconds);
}
if(debug) { Serial.print("Sunset at "); }
if(sethourlength == 11){
if(debug) {
Serial.print("0");
Serial.print(contents[foundsunset].charAt(8));
}
sunsetseconds = ((contents[foundsunset].charAt(8) - '0') * 3600);
}
else{
if(debug) {
Serial.print(contents[foundsunset].charAt(8));
Serial.print(contents[foundsunset].charAt(9));
}
sunsetseconds = ((contents[foundsunset].charAt(8) - '0') * 10 * 3600) + ((contents[foundsunset].charAt(9) - '0') * 3600);
}
if(debug) { Serial.print(":"); }
if(setminutelength == 12){
if(debug) {
Serial.print("0");
Serial.println(contents[foundsunset + 1].charAt(10));
}
sunsetseconds = sunsetseconds + ((contents[foundsunset + 1].charAt(10) - '0') * 60);
}
else{
if(debug) {
Serial.print(contents[foundsunset + 1].charAt(10));
Serial.println(contents[foundsunset + 1].charAt(11));
}
sunsetseconds = sunsetseconds + ((contents[foundsunset + 1].charAt(10) - '0') * 10 * 60) + ((contents[foundsunset + 1].charAt(11) - '0') * 60);
}
if(debug) {
Serial.print("Seconds from midnight to SUNSET: ");
Serial.println(sunsetseconds);
}
//validData = false; // reset flag not needed here as it is done in V2
}
else{
validData = false;
if(debug) { Serial.println("Data is INVALID, check API"); }
}
if(debug) { Serial.println("Connecting Blynk"); }
//Blynk.connect();
}
void setup() {
Serial.begin(115200);
Serial.println("\nStarting");
Blynk.begin(authstr, ssidstr, passstr, serverstr);
int mytimeout = millis() / 1000;
while (Blynk.connect() == false) { // wait here for upto 10s until connected to the server
if((millis() / 1000) > mytimeout + 8){ // try to connect to the server for less than 9 seconds
break; // continue with the sketch regardless of connection to the server
}
}
Serial.print("IP address: ");
Serial.println(WiFi.localIP());
ArduinoOTA.setHostname(OTAhost); // for local OTA updates
ArduinoOTA.begin();
timer.setInterval(5000L, reconnectBlynk); // check every 5s if still connected to server
terminalW.println();
terminalW.print("Local IP: ");
terminalW.println(WiFi.localIP());
terminalW.flush();
getrawdata(); // now also done with V2 momentary switch
gotValidData();
}
void loop() {
if (Blynk.connected()) { // to ensure that Blynk.run() function is only called if we are still connected to the server
Blynk.run();
}
timer.run();
ArduinoOTA.handle(); // for local OTA updates
yield();
}
2 Likes
Great work! Currently I’m busy building a lighting system for Lego houses and this might come in handy there too …
1 Like
The great thing about your code is that it can be easily adapted for all the many, great API’s that are currently available.
I did think about making the sketch more generic whereby you would enter a keyword in the sketch or say Blynk Terminal such as “Sunrise”, an offset for the index where the data for the keyword is located and a few integer variables to pull out the data i.e. to replace 1, 8 for example in the following line:
if(inputString.substring(1, 8) == "sunrise"){
1 Like
Hmm, you could be on to something there. I think maybe even more generic would be something to parse XML files. Since they are probably the standard of API’s. I just don’t have a lot of experience using XML files. The trick would probably to determine how many levels of parameters you have and how to store them dynamically in arrays and such. I think C or C++ would be better for this since they handle arrays a lot better than a simple MCU.
Btw, I see you use chars and strings and such to get the time’s with leading zero’s and whatnot. I highly recommend using UNIX time. It’s so much easier to deal with once you get the hang of it. And it saves a lot of memory in the end.
For reference, this is how I process the captured data:
/*
* Sunrise/sunset timer via API for use with RTC Widget in Blynk
*
* (C) 2016 B. Visee, info@lichtsignaal.nl
* This code is licensed under MIT license
*/
bool procesdata()
{
for(int i=0;i<NUM_LEDS;i++)
{
leds[i] = CRGB::Pink;
FastLED.show();
}
// Temp variable for storing dates and times
String parameter[10][2];
// Remove first {"results":
int firstOpening = useFullData.indexOf('{');
int secondOpening = useFullData.indexOf('{', firstOpening + 1);
useFullData.remove(0, secondOpening + 1);
// Remove ending ,"status":"OK"}
int firstClosing = useFullData.indexOf('}');
useFullData.remove(firstClosing);
if(debug) { Serial.println(useFullData); }
// Put parameters and values into the temporary array parameters[][]
for(int i=0;i<10;i++)
{
firstOpening = useFullData.indexOf('\"');
secondOpening = useFullData.indexOf('\"', firstOpening + 1);
parameter[i][0] = useFullData.substring(firstOpening+1, secondOpening);
useFullData.remove(0, secondOpening+1);
firstOpening = useFullData.indexOf('\"');
secondOpening = useFullData.indexOf('\"', firstOpening + 1);
parameter[i][1] = useFullData.substring(firstOpening+1, secondOpening);
useFullData.remove(0, secondOpening+1);
if(debug)
{
Serial.print("Parameter: ");
Serial.print(parameter[i][0]);
Serial.print(" contains: ");
Serial.println(parameter[i][1]);
}
}
// Convert times into usable UNIX times
firstOpening = parameter[0][1].indexOf(':');
secondOpening = parameter[0][1].indexOf(':', firstOpening+1);
int thirdOpening = parameter[0][1].indexOf(' ', secondOpening+1);
// Store all the times in the time variables
for(int i=0;i<10;i++)
{
if(i != 2)
{
int tempHour = parameter[i][1].substring(0,firstOpening).toInt();
int tempMinute = parameter[i][1].substring(firstOpening+1,secondOpening).toInt();
int tempSecond = parameter[i][1].substring(secondOpening+1,thirdOpening).toInt();
// Correct for 24h clock
if(parameter[i][1].endsWith("PM") )
{
tempHour = tempHour + 12;
}
tmElements_t tm;
tm.Hour = tempHour;
tm.Minute = tempMinute;
tm.Second = tempSecond;
tm.Day = day(currentTime);
tm.Month = month(currentTime);
tm.Year = year(currentTime) - 1970;
switch(i)
{
case 0:
sunrise = myTZ.toLocal(makeTime(tm), &tcr);
sunriseOriginal = myTZ.toLocal(makeTime(tm), &tcr); // Keep backup of original times for custom times
if(debug) { Serial.println(sunrise); }
break;
case 1:
sunset = myTZ.toLocal(makeTime(tm), &tcr);
sunsetOriginal = myTZ.toLocal(makeTime(tm), &tcr); // Keep backup of original times for custom times
if(debug) { Serial.println(sunset); }
break;
case 3:
solar_noon = myTZ.toLocal(makeTime(tm), &tcr);
if(debug) { Serial.println(solar_noon); }
break;
case 4:
civil_twilight_begin = myTZ.toLocal(makeTime(tm), &tcr);
if(debug) { Serial.println(civil_twilight_begin); }
break;
case 5:
civil_twilight_end = myTZ.toLocal(makeTime(tm), &tcr);
if(debug) { Serial.println(civil_twilight_end); }
break;
case 6:
nautical_twilight_begin = myTZ.toLocal(makeTime(tm), &tcr);
if(debug) { Serial.println(nautical_twilight_begin); }
break;
case 7:
nautical_twilight_end = myTZ.toLocal(makeTime(tm), &tcr);
if(debug) { Serial.println(nautical_twilight_end); }
break;
case 8:
astronomical_twilight_begin = myTZ.toLocal(makeTime(tm), &tcr);
if(debug) { Serial.println(astronomical_twilight_begin); }
break;
case 9:
astronomical_twilight_end = myTZ.toLocal(makeTime(tm), &tcr);
if(debug) { Serial.println(astronomical_twilight_end); }
break;
}
}
}
if(debug) { Serial.println("Converted all times to correct timezone"); }
// Check if both hours of sunrise and sunset are filled, we assume it's ok then.
if( (hour(sunrise) != 0) && (hour(sunset) != 0) )
{
for(int i=0;i<NUM_LEDS;i++)
{
leds[i] = CRGB::Black;
FastLED.show();
}
return 1;
}
else
{
// Set LED's in error state
for(int i=0;i<NUM_LEDS;i++) { leds[i] = CRGB::Red; FastLED.show(); }
return 0;
}
}
1 Like
@Lichtsignaal for my requirement it is only really the seconds from midnight for sunrise and sunset that I require, rather than the time per se. As you have probably seen with debug set as 1 it does show the actual times in Serial Monitor but only the sunriseseconds andsunsetseconds are used in the Blynk project and passed to the Terminal widget.
I think my use of Strings may be because many moons ago I started with GW Basic and have always found them easier to work with.
It was not ideal for Arduino’s and very much frowned upon but now we have so much memory to work with on the ESP’s String use is more acceptable.
When I have a minute I will study your code as I’m sure there are things I will find useful, thanks.
Yr welcome. It all depends on what you want to achieve of course. I’ve noticed the calculations with time are easier with UNIX time, but if your input is limited, you could do without
I figured Unix time was a pain, but it is really easy with the Time library if you take a little time (pun intended) to study it 
@Costas @Lichtsignaal this request -
http://api.sunrise-sunset.org/json?lat=36.7201600&lng=-4.4203400&date=2016-08-25
returns
{"results":{"sunrise":"5:43:55 AM","sunset":"6:55:05 PM","solar_noon":"12:19:30 PM","day_length":"13:11:10","civil_twilight_begin":"5:17:12 AM","civil_twilight_end":"7:21:47 PM","nautical_twilight_begin":"4:45:18 AM","nautical_twilight_end":"7:53:42 PM","astronomical_twilight_begin":"4:12:05 AM","astronomical_twilight_end":"8:26:55 PM"},"status":"OK"}
Do you use all this fields or just 1 specific? Like sunrise?
@Dmitriy for me it would just be sunrise and sunset at the moment as any “twilight” adjustments could be done with a slider. Ideally though all fields from all API’s should be available. Not an easy task I know.
Perhaps I should rephrase that. Say X number of fields to be available from each API. So all fields technically being available but a limitation on how many can be used within Blynk at any one time. As you will know some API’s have hundreds of fields and it wouldn’t be practical to be working on all of them within Blynk.
I parse all fields, so I can use anything which it returns. You can choose wether to use it or not, but everything is there, yes
Well, I was thinking on how your code could be simplified with Blynk. Because when I look in @Costas parsing code I feel pain .
1 Like
@Dmitriy yes painful to a real coder. Please note my parsing is for api.wunderground.com not api.sunrise-sunset.org and wunderground has lots more stuff in addition to the astronomy api.
This is not because of your code, this parsing in general is complex and I believe it is impossible task for most of Blynkers.
You would be in agony then if you saw what we do when we parse Blynk’s “project” api.
1 Like
Well, how far do you want to go? It may be enough to write a basic parser for “known” API’s or basic XML feeds.