Local Server - Sockets/HTTPS API/Memory

Hi @Dmitriy

I’m running local server v0.23.3
I am using ESP8266 (Wemos Mini) and sending BLYNK HTTPS API calls from it to my Blynk Local Server.

I can see (and assume) that the HTTPS connection setup between my ESP and the Blynk Local server is KeepAlive (on) when I make client.connect() calls from my ESP ; I also notice when I send a client.stop() it seems not to close the connection and my HEAP is not decreased accordingly; infact just as a test if I make multiple client.connect commands my HEAP keeps decreasing even though i have a corresponding client.close() for each client.connect. My questions are:

  1. does Blynk server immediately close connection when a client.stop() is sent and is memory returned
  2. How can I adjust the KeepAlive time - does it relate to any of the settings as per below screenshot

NB: I can post some code examples if required to explain situation further.

Hi. What HEAP do you mean here?

In all examples I see client.stop() but not client.close(). Maybe this is a reason? @vshymanskyy should know better.

apologies, yes I mean client.stop() . my mistake on that one.
The heap I refer to is the ESP.getFreeHeap()

Can you pls comment on what the two values below specific relate too, i.e. if my sketch is setting up client.connect() which timeout is used by Blynk Server ?

In order to answer your question I need to know what client.stop() does. But I don’t. There 2 ways to close HTTPS connection :

  1. Client can send connection: close header and in that case server should close connection;
  2. Client just closes socket and specific TCP acknowledgement is sent. In that case server has no influence on client. In other words connection is closed on client side, so client should free what it had allocated.

perhaps this can help to articulate my issues:

The below code [CODE 1 EXAMPLE] works well as it opens connection ONCE then continues to send HTTPS API commands to local Blynk Server. The ESP.getFreeHeap() remains static and my little WEMOS keeps sending HTTPS API commands to my local Blynk Server without any problems.

The seconds code example [CODE 2 EXAMPLE] keeps opening and closing HTTP connections and eventually the ESP.getFreeHeap() goes to zero the my little WEMOS crashes and restarts. I don’t know why this is happening.

CODE 1 EXAMPLE : this works well…no crashes, and ESP heap remains the same

#include <ESP8266WebServer.h>
#include <DNSServer.h>
#include <ESP8266WiFi.h>
#include <WiFiClientSecure.h>
#include <ESP8266mDNS.h>

// setup for https calls
const int httpsPort = 9443;
// const int httpsPort = 443;
const char* host_toshiba = "xxxxx.com";



// Your WiFi credentials.
// Set password to "" for open networks.
char ssid[] = "xxxx";
char pass[] = "xxxx";


int counter = 0;

WiFiClientSecure client;

void setup() {
  // put your setup code here, to run once:
  int waittime = millis() + 10000;

  Serial.begin(115200);
  Serial.println("\n Starting");

 WiFi.begin(ssid, pass);          // use this code if we want to hardwire an access point
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
    if (waittime < millis()) { break; }
  }
  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());


} // end setup


//=========== run_http ==============

void run_http (int x)
{

// WiFiClientSecure client;


    if (!client.connected())
    {
      client.connect(host_toshiba, httpsPort);
    }

       client.print(String("GET ") + "/xxxx/update/V4?value=" + String (x) + " HTTP/1.1\r\n" +
               "Host: " + host_toshiba + "\r\n" +
               "User-Agent: BuildFailureDetectorESP8266\r\n" +
               "Connection: close\r\n\r\n");

          while(client.available()) {
          Serial.write(client.read());
       }
} // end run_http

void check_heap()
{
    int heap_available = ESP.getFreeHeap();
      client.print(String("GET ") + "/xxxxx/update/V8?value=" + String(heap_available) + String(",")+" HTTP/1.1\r\n" +
               "Host: " + host_toshiba + "\r\n" +
               "User-Agent: BuildFailureDetectorESP8266\r\n" +
               "Connection: KeepAlive Off" + "Connection: close\r\n\r\n");

   while(client.available()) {
          Serial.write(client.read());
       }

      client.print(String("GET ") + "/xxxxx/update/V9?value=" + String(counter++) +" HTTP/1.1\r\n" +
               "Host: " + host_toshiba + "\r\n" +
               "User-Agent: BuildFailureDetectorESP8266\r\n" +
               "Connection: KeepAlive Off" + "Connection: close\r\n\r\n");

        while(client.available()) {
          Serial.write(client.read());
       }

     Serial.println (ESP.getFreeHeap());

}


void loop() {
  // put your main code here, to run repeatedly:

 for (int x=0; x < 40; x++) {
   run_http (x);
 }

 check_heap();

 if (client.connected())
 { Serial.println ("client still connected...");}

 if (!client.connected())
 { Serial.println ("client not connected...."); }

}

CODE 2 EXAMPLE : eventually crashes my WEMOS as ESP heaps keeps decrementing each time client.connect() is called even though I close connection with client.stop()

#include <ESP8266WebServer.h>
#include <DNSServer.h>
#include <ESP8266WiFi.h>
#include <WiFiClientSecure.h>
#include <ESP8266mDNS.h>

// setup for https calls
const int httpsPort = 9443;
// const int httpsPort = 443;
const char* host_toshiba = "xxxx.com";
// const char* host_fileserver1 = "192.168.0.251";

// Your WiFi credentials.
// Set password to "" for open networks.
char ssid[] = "xxx";
char pass[] = "xxx";


int counter = 0;

WiFiClientSecure client;

void setup() {
  // put your setup code here, to run once:
  int waittime = millis() + 10000;

  Serial.begin(115200);
  Serial.println("\n Starting");

 WiFi.begin(ssid, pass);          // use this code if we want to hardwire an access point
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
    if (waittime < millis()) { break; }
  }
  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());


} // end setup


//=========== run_http ==============

void run_http (int x)
{

// WiFiClientSecure client;


    if (!client.connected())
    {
      client.connect(host_toshiba, httpsPort);
    }

       client.print(String("GET ") + "/xxxxxxx/update/V4?value=" + String (x) + " HTTP/1.1\r\n" +
               "Host: " + host_toshiba + "\r\n" +
               "User-Agent: BuildFailureDetectorESP8266\r\n" +
               "Connection: close\r\n\r\n");

          while(client.available()) {
          Serial.write(client.read());
       }
} // end run_http

void check_heap()
{
    int heap_available = ESP.getFreeHeap();
      client.print(String("GET ") + "/xxxxx/update/V8?value=" + String(heap_available) + String(",")+" HTTP/1.1\r\n" +
               "Host: " + host_toshiba + "\r\n" +
               "User-Agent: BuildFailureDetectorESP8266\r\n" +
               "Connection: KeepAlive Off" + "Connection: close\r\n\r\n");

   while(client.available()) {
          Serial.write(client.read());
       }

      client.print(String("GET ") + "/xxxxx/update/V9?value=" + String(counter++) +" HTTP/1.1\r\n" +
               "Host: " + host_toshiba + "\r\n" +
               "User-Agent: BuildFailureDetectorESP8266\r\n" +
               "Connection: KeepAlive Off" + "Connection: close\r\n\r\n");

        while(client.available()) {
          Serial.write(client.read());
       }

     Serial.println (ESP.getFreeHeap());

}


void loop() {
  // put your main code here, to run repeatedly:

 for (int x=0; x < 40; x++) {
   run_http (x);
 }

 check_heap();


 if (client.connected())
 { Serial.println ("client still connected...");
   client.flush();   // ADDED to EXAMPLE 2
   client.stop();   // ADDED to EXAMPLE 2 ....close connection
}

 if (!client.connected())
 { Serial.println ("client not connected...."); }

}

@Costas adding Costas as I know you have a lot of experience with WEMOS / ESP8266 devices.
Costas my ESP HEAP keeps reducing everytime I call call.connect() and eventually resetting - even though I call a client.stop() for each client.connect() call.

I would have presumed a client.stop() would close the connection and returned any used memory back to the WEMOS…but jugging from the ESP.gerFreeHeap() call it does not.

See CODE 2 EXAMPLE for more context.

@mars in the second example try commenting out both Serial.println() statements in the loop() and see if this fixes the heap. If it does, try commenting out just the first Serial.println().

Hi - no it didn’t help. I think fundamentally the Blynk server handles connections differently or simply ignores the “Connection : close”

@Dmitriy I tried the following code published by Ivan Grokhotkov (https://gist.github.com/igrr/816523b463bd7d045b77) and ESP heap that is printed our to serial remain stable.

as per below:

/*
 *  HTTP over TLS (HTTPS) example sketch
 *
 *  This example demonstrates how to use
 *  WiFiClientSecure class to access HTTPS API.
 *  We fetch and display the status of
 *  esp8266/Arduino project continous integration
 *  build.
 *
 *  Created by Ivan Grokhotkov, 2015.
 *  This example is in public domain.
 */

#include <ESP8266WiFi.h>
#include <WiFiClientSecure.h>
#include <time.h>

const char* ssid = "            ";
const char* password = "              ";

const char* host = "api.github.com";
const int httpsPort = 443;

// Use web browser to view and copy
// SHA1 fingerprint of the certificate
const char* fingerprint = "CF 05 98 89 CA FF 8E D8 5E 5C E0 C2 E4 F7 E6 C3 C7 50 DD 5C";

void setup() {
  Serial.begin(115200);
  Serial.println();
  Serial.print("connecting to ");
  Serial.println(ssid);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
  configTime(3600, 0, "pool.ntp.org");
}

void loop() {
  // Use WiFiClientSecure class to create TLS connection
  WiFiClientSecure client;
  if (!client.connect(host, httpsPort)) {
    Serial.println("connection failed");
    return;
  }


  String url = "/repos/esp8266/Arduino/commits/esp8266/status";

  client.print(String("GET ") + url + " HTTP/1.1\r\n" +
               "Host: " + host + "\r\n" +
               "User-Agent: BuildFailureDetectorESP8266\r\n" +
               "Connection: close\r\n\r\n");

//  Serial.println("waiting for connection to close");
  while (client.connected() || client.available()) {
    client.read();
  }
  time_t t = time(nullptr);
  Serial.printf("time: %s\tHeap: %d\n", ctime(&t), ESP.getFreeHeap());
  delay(5000);
}

THEN given the above code works well;
I substituted the https request with one to my local Blynk Server and it just gets caught in while statement loop in loop() [see BELOW CODE]

Dmitriy I assume the Blynk Server does not process the “Connection : close” command ? or something weird is happening I can’t explain. Hoping you can help ?

My Blynk Version of Ivan’s code that crashes WEMOS:

*
 *  HTTP over TLS (HTTPS) example sketch
 *
 *  This example demonstrates how to use
 *  WiFiClientSecure class to access HTTPS API.
 *  We fetch and display the status of
 *  esp8266/Arduino project continous integration
 *  build.
 *
 *  Created by Ivan Grokhotkov, 2015.
 *  This example is in public domain.
 */

#include <ESP8266WiFi.h>
#include <WiFiClientSecure.h>
#include <time.h>

const char* ssid = "xxxx";
const char* password = "xxxx";

const char* host = "blynk.xxxx.com";
const int httpsPort = 9443;
// const int httpsPort = 443;

// Use web browser to view and copy
// SHA1 fingerprint of the certificate
const char* fingerprint = "CF 05 98 89 CA FF 8E D8 5E 5C E0 C2 E4 F7 E6 C3 C7 50 DD 5C";

void setup() {
  Serial.begin(115200);
  Serial.println();
  Serial.print("connecting to ");
  Serial.println(ssid);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
  configTime(3600, 0, "pool.ntp.org");
}

int x = 0;

void loop() {
  // Use WiFiClientSecure class to create TLS connection
  WiFiClientSecure client;
  if (!client.connect(host, httpsPort)) {
    Serial.println("connection failed");
    return;
  }

  
  String url = "/xxxx/update/V4?value=" + String ((x++));
  Serial.println (F("I'm before client print"));
  client.print(String("GET ") + url + " HTTP/1.1\r\n" +
               "Host: " + host + "\r\n" +
               "User-Agent: BuildFailureDetectorESP8266\r\n" +
               "Connection: close\r\n\r\n");

//  Serial.println("waiting for connection to close");
    Serial.println (F("I'm after client print"));
  while (client.connected() || client.available()) {    // **GETS CAUGHT IN LOOP**
    client.read();
  }
    Serial.println (F("I'm after while loop"));
  time_t t = time(nullptr);
  Serial.printf("time: %s\tHeap: %d\n", ctime(&t), ESP.getFreeHeap());
//  delay(5000);
}

That’s correct.

@mars bug has been added to GitHub hopefully for release in milestone 0.24.0 at https://github.com/blynkkk/blynk-server/issues/559

Presumably you will be changing the fingerprint in your sketch from that of api-github.com to blynk-cloud.com

1 Like

thats good news thank, Costas & @Dmitriy
Dmitriy if you could kindly educate me (and the Blynk community) how these below server.properties effect HTTPS API Blynk calls or for that matter a bit more context on what and how they are used would be great :wink:

@Costas On the fingerprint - yes; i need y own fingerprint - to be honest I’m struggling to get that to work at this stage with my local server setup, I’ve created an SSL certificate using the instructions on blynk.cc but still I can’t get the whole fingerprint to work - still needs further education on my part.

@Dmitriy hi - thanks for the fix to support connection close in local server 0.24.0 - it works well.
I wanted to understand how long the local blynk server will hold the connection before it times-out and closes ?
Are the below attributes in server.properties somehow related - could you be kind enough to explain the use of the below properties - I’m trying to understand if HTTP API’s use the Application Socket or Hardware socket ?

For HTTP API - no. For HTTP API there is no Idle connection handlers. In order words connection is keep opened until it is closed by client.