How To check the Blynk Server Inbound Data

[SmsAlert]

Hey Blynkers, wishing you all an awesome 2022.

I have been working on a new project and have hit a snag.

I am testing a GSM module that is running uPython and am having issues connecting the device to the cloud.

The following info is what is being used:

1. Using the Blynk IoT app and platform (latest and greatest)
2. My firmware is fully functional on the ESP32 (Blynk Library), so used this library
3. EC600 Cellular module running uPython.
4. SIM card and network related connections are all in order.
5. Looked around the forum - if I missed something there happy to delete this thread.

Issue:

1. I established the socket connection (non-secure) and connect to the blynk platform, which also returns successful.
2. I send the Auth Token and this returns as “Invalid Token”

Debug trace here:
—> Network Connection Status stagecode = 3, subcode = 1
—> Network Status: ([5, 20513, 5458206, 7, 0, 0], [5, 20513, 5458206, 7, 0, 0])
—> Check IP Address: (1, 2, [1, 0, ‘10.102.120.253’, ‘193.181.246.58’, ‘193.181.246.57’])
—> Connect To Server
—> Blynk Log: Connect: Server: “blynk-cloud.com” : Port: “80”
—> Blynk Log: Connect: Socket Setup
—> Blynk Log: Connect: Non Secure Socket
—> Blynk Log: Connect: Socket Set Time Out
—> Blynk Log: Socket Status: Connection Established
—> Blynk Log: Login: Started
—> Blynk Log: Connect: Last Received: 891583845 :: Last Send: 0 :: Last Ping: 0
—> Blynk Log: Login: Token: 7KzlYaWqvL01Lvg1mMKML8P0hl0zGk2r
—> Blynk Log: > Sending CMD: 29
—> Blynk Log: > Sending ARGS: 7KzlYaWqvL01Lvg1mMKML8P0hl0zGk2r
—> Blynk Log: > Sending KWARGS:
—> Blynk Log: > Sending ID: 1
—> Blynk Log: > Outbound Data Packet CMD: 29, ID: 1, Data: 7KzlYaWqvL01Lvg1mMKML8P0hl0zGk2r
—> Blynk Log: > Outbound Data Packet: b’ \x1d\x00\x01\x00 7KzlYaWqvL01Lvg1mMKML8P0hl0zGk2r’
—> Blynk Log: > Outbound Data b’\x1d\x00\x01\x00 7KzlYaWqvL01Lvg1mMKML8P0hl0zGk2r’
—> Blynk Log: < Check For Data From Server
—> Blynk Log: < Initial Time: 891584017
—> Blynk Log: < Timeout Time: 891586017
—> Blynk Log: < Server Data: b’\x00\x00\x01\x00\t’
—> Blynk Log: < Inbound Data Packet: CMD: 0, ID: 1,STA: 9
—> Blynk Log: < BIN Data: b’\x00\x00\x01\x00\t’
—> Blynk Log: < Server Acknowledge: CMD: 0 ID: 1 Response Else: 9
—> Blynk Log: Execute Event: invalid_auth

I have checked with the module manufacture to see if there is any corrupt data during the processes and they advised that the data packet is sent correctly. Screen shot of their response, validating the sent data:

Questions:

1. Is there a way to see the incoming connection (on the platform) from the field device and check the inbound data / connection?
2. Am I missing something on my side when communicating with the server?

[PeteKnight]

@SmsAlert

Using the Blynk IoT app and platform

@SmsAlert

Blynk Log: Connect: Server: “blynk-cloud.com” : Port: “80”

That’s the legacy server. The Blynk IoT server is blynk.cloud

Auth tokens are not shared amongst the two environments so the Legacy cloud server is correctly reporting that it doesn’t recognise your Blynk IoT auth token.

Pete.

[SmsAlert]

Thanks Pete, seemed to have found part of the issue, will need to go back to the module vendor and do a few more checks, will update here on the findings

[SmsAlert]

Update:
I have managed to get the cellular module connected on a non secure connection via port 80.
found that the cellular module had a DNS resolution issue so managed to connect on “ny3.blynk.cloud”

Next question:
I get the following response from the server: b’\x17\x03\x03\x00\x1d’, the 0x17 (decimal 23), is there a description for the server responses, below is code that I have managed to salvage from the arduino library and gives a ‘basic’ headline, but as you will see there are response codes that are missing, any way of getting more details on these response codes?

Not sure what the “STA” stands for… any ideas?

#------------------------------------------------------------------------------
#	Global Defines
#------------------------------------------------------------------------------
MSG_RSP = const(0)										# Server Response to Received Message
MSG_LOGIN = const(2)									# Login Command
MSG_PING  = const(6)									# Ping Command
MSG_TWEET = const(12)									# Not Tested
MSG_EMAIL = const(13)									# Email Command
MSG_NOTIFY = const(14)									# Notification Command (APP Only)
MSG_BRIDGE = const(15)
MSG_HW_SYNC = const(16)									# Syncronise Hardware Command
MSG_INTERNAL = const(17)								# Internal Message Command
MSG_SEND_SMS = const(18)								# Not Tested
MSG_PROPERTY = const(19)								# Property Command
MSG_HW = const(20)										# Hardware Command
MSG_HW_LOGIN = const(29)								# Login Command
MSG_REDIRECT  = const(41)  								# TODO: not implemented
MSG_DBG_PRINT  = const(55) 								# TODO: not implemented
MSG_EVENT_LOG = const(64)								# Event logging Command (Must be defined on the platform and then Linked)
MSG_EVENT_CLEAR = const(65)								# Not Tested
# Server Responses
STA_QUOTA_LIMIT_EXCEPTION = const(1)					# TODO: not implemented
STA_ILLEGAL_COMMAND = const(2)							# TODO: not implemented
STA_NOT_REGISTERED = const(3)							# TODO: not implemented
STA_ALREADY_REGISTERED = const(4)						# TODO: not implemented
STA_NOT_AUTHENTICATED = const(5)						# TODO: not implemented
STA_NOT_ALLOWED = const(6)								# TODO: not implemented
STA_DEVICE_NOT_IN_NETWORK = const(7)					# TODO: not implemented
STA_NO_ACTIVE_DASHBOARD = const(8)						# TODO: not implemented
STA_INVALID_TOKEN = const(9)							# TODO: not implemented
STA_ILLEGAL_COMMAND_BODY = const(11)					# TODO: not implemented
STA_GET_GRAPH_DATA_EXCEPTION = const(12)				# TODO: not implemented
STA_NTF_INVALID_BODY = const(13)						# TODO: not implemented
STA_NTF_NOT_AUTHORIZED = const(14)						# TODO: not implemented
STA_NTF_ECXEPTION= const(15)							# TODO: not implemented
STA_TIMEOUT = const(16)									# TODO: not implemented
STA_NO_DATA_EXCEPTION = const(17)						# TODO: not implemented
STA_DEVICE_WENT_OFFLINE = const(18)						# TODO: not implemented
STA_SERVER_EXCEPTION = const(19)						# TODO: not implemented
STA_NOT_SUPPORTED_VERSION = const(20)					# TODO: not implemented
STA_ENERGY_LIMIT = const(21)							# TODO: not implemented
STA_OPERATION_ACCEPT = const(23)						# TODO: not implemented
STA_OPERATION_DECLINE = const(24)						# TODO: not implemented
STA_SUCCESS= const(200)									# Data Uploaded Successfully
# Connection Status	
DISCONNECTED = const(0)
CONNECTING = const(1)
CONNECTED = const(2)
AUTHENTICATED = const(3)
#------------------------------------------------------------------------------

[PeteKnight]

@SmsAlert

I get the following response from the server: b’\x17\x03\x03\x00\x1d’, the 0x17 (decimal 23)

Well, that means…

@SmsAlert

STA_OPERATION_ACCEPT = const(23)

@SmsAlert

there are response codes that are missing

If you mean that there are some gaps in the response codes (10 & 22) then I guess that these are deprecated codes that weren’t reused.
Certainly response code 22 (Dec) used to mean FACEBOOK_USER_LOGIN_WITH_PASS but that’s now retired as FaceBook logins are no longer used.

That information comes from here:

GitHub - Peterkn2001/blynk-server: This is a fork of the now discontinued and...

and the “Possible Response Codes” link. Obviously these responses were for the Legacy local server, and some new codes have been added for IoT as well as at least one being retired.

Pete.

[SmsAlert]

@SmsAlert

STA_OPERATION_ACCEPT = const(23)

… ok, that was a real stupid mistake, missed that one, thanks Pete.

@PeteKnight

That information comes from here:

github.com

GitHub - Peterkn2001/blynk-server: This is a fork of the now discontinued and… - Blynk protocol

This is a fork of the now discontinued and unsupported Blynk Legacy Local Server documentation and releases. - GitHub - Peterkn2001/blynk-server: This is a fork of the now discontinued and unsuppor…

this was the old server if I understand correctly, is there updated info for the new servers, I am trying to see why the ssl options are not working

Response from Non Secure: < Server Data: b’\x00\x00\x01\x00\xc8’

Response when I use ssl (only on cellular module, ESP32 is same for both secure and non secure connections): < Server Data: b’\x17\x03\x03\x00\x1d\x00\x00\x00\x00\x00\x00\x00\x01\xb4\xcfQ\x94\x95\x92G\x9e7\xf1\xdb;\\x1e\xfclN\xa0\xd8\xe8\xc8’

The first 5 bytes are always the same value b’\x17\x03\x03\x00\x1d’, so want to try decode the message from the server:
Byte 1: STA_OPERATION_ACCEPT
Byte 2/3: 771 (this is the ID byte)
Byte 4: 29 (not sure what this is or represents)
Byte 5: 0 (this is the success byte should be 0xC8)
Rest of the Data: mmmm… not sure how to decode this info

I have sent a trace log to the cellular module manufacturer to see if there are any funnies on the module side, but just seems odd that the responses are so different betwene the secure and non secure on the cellular module, yet on the ESP32 they are the same.

[PeteKnight]

@SmsAlert

is there updated info for the new servers

No, not as far as I know.

TBH, if you can run C++ on this hardware then I’d recommend doing that, with debugging turned on. You’ll see all of the outbound commands and inbound responses and you’ll be able to make more sense of them.

Also, are you sure that the hardware or the carrier supports SSL?

Pete.

[SmsAlert]

@PeteKnight

TBH, if you can run C++ on this hardware then I’d recommend doing that, with debugging turned on. You’ll see all of the outbound commands and inbound responses and you’ll be able to make more sense of them.

Can only run C or uPython… thought the uPython would have been a quick port from the ESP32. have it working like a charm on the ESP32 and starting with prototypes in Feb.

@PeteKnight

Also, are you sure that the hardware or the carrier supports SSL?

Will only have the feedback from the module vendor on the trace logs tomorrow, will check with the carriers as well.

Thanks for the help, will update again once I have the feedback

[SmsAlert]

Update 1: Still resolving the SSL issue with the module vendor, may take a few more days so will update next week again.

[SmsAlert]

Update 2:
After a few days of up and down with the module vendor we are stuck. They have tested the code and have a few issues with parsing the data and data analysis between the non SSL and SSL options in uPython.

Is there a way to get a server side trace of the inbound connection from the edge device?

yes I understand that this info is not available in a public forum, so a PM will be the way.

If this is not an option, then please suggest something that I can go test and try.

Maybe I have missed something in the SSL connection on the ESP32 that might not be “apparent” on the cellular module (Oh wouldn’t it be nice if we could just move code around… )

here is the Python code that I use for the “connect” routine

	def connect(self):
		self.log('Connect: Server: "%s" : Port: "%d"' % (self.server, self.port))
		try:
			self.socket = None
			self.conn = None
			self.log('Connect: socket.socket(socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_TCP)')
#			self.socket = socket.socket()
			self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_TCP)
#			self.socket = socket.socket(socket.AF_INET, socket.socket.SOCK_RAW)
			self.log('connect(socket.getaddrinfo(self.server, self.port)[0][-1])')
			self.socket.connect(socket.getaddrinfo(self.server, self.port)[0][-1])
#			self.socket.connect(socket.getaddrinfo(self.server, self.port, 0, SOCK_STREAM)[0][-1])
#			self.socket.connect(socket.getaddrinfo(self.server, self.port))
			self.log('Connect: Socket Setup')
#			try:
#				self.log('Connect: socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)')
#				self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
#				self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
#				self.log('Connect: Set Socket Options')
#			except:
#				pass
			if self.insecure:										# NON Secure Connection
				self.conn = self.socket
				self.log('Connect: Non Secure Socket')
			else:													# Secure Connection	
				self.log('Connect: Secure Socket')
				try:
					import ussl
					ssl_context = ussl
					self.log('Connect: Import uSSL')
				except Error as ierr:
					self.log('Connect: uSSL Import Error: ', str(ierr))
					import ssl
					ssl_context = ssl.create_default_context()
					self.log('Connect: Import SSL')
				try:
					self.conn = ssl_context.wrap_socket(self.socket, server_hostname=self.server)
					self.log('Connect: SSL Context Wrap')
				except Error as err:
					self.log('Connect: SSL Context Error: ', str(err))
					self.disconnect()
					self.log('Connect: Disconnected')
					return
			try:
				self.conn.settimeout(SOCK_TIMEOUT)
				self.log('Connect: Socket Set Time Out')
			except:
				self.socket.settimeout(SOCK_TIMEOUT)
				self.log('Connect: Socket Set Time Out')
			try:
				while True:
					if self.conn.getsocketsta() == 4:
						self.log('Socket Status: CONN Connection Established')
						Protocol.connect(self)
						break
					else:
						self.state = DISCONNECTED
			except:
				if self.socket.getsocketsta() == 4:
					self.log('Socket Status: Socket Connection Established')
					Protocol.connect(self)
				else:
					self.state = DISCONNECTED
		except:
			self.log('Connect Exception: Close The Socket')
			try:													# Close SSL Connection (Insecure = True)
				self.conn.close()
				self.socket.close()
				self.log('Connect: Both Closed')
			except:													# Close Socket (Insecure = False)
				try:
					self.socket.close()
					self.log('Connect: Socket Closed')
				except:
					self.log('Connect: Socket Closed Exception')

[SmsAlert]

Question: Are any of the below SSL algorithms used (Edit: on the Blynk Server)

//Algorithm Suite
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca9)
TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xccaa)
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca8)
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)
TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x009f)
TLS_ECDHE_ECDSA_WITH_AES_256_CCM (0xc0ad)
TLS_DHE_RSA_WITH_AES_256_CCM (0xc09f)
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024)
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)
TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (0x006b)
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x0039)
TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8 (0xc0af)
TLS_DHE_RSA_WITH_AES_256_CCM_8 (0xc0a3)
TLS_ECDHE_ECDSA_WITH_CAMELLIA_256_GCM_SHA384 (0xc087)
TLS_ECDHE_RSA_WITH_CAMELLIA_256_GCM_SHA384 (0xc08b)
TLS_DHE_RSA_WITH_CAMELLIA_256_GCM_SHA384 (0xc07d)
TLS_ECDHE_ECDSA_WITH_CAMELLIA_256_CBC_SHA384 (0xc073)
TLS_ECDHE_RSA_WITH_CAMELLIA_256_CBC_SHA384 (0xc077)
TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA256 (0x00c4)
TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA (0x0088)
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x009e)
TLS_ECDHE_ECDSA_WITH_AES_128_CCM (0xc0ac)
TLS_DHE_RSA_WITH_AES_128_CCM (0xc09e)
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 (0xc023)
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)
TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (0x0067)
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x0033)
TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8 (0xc0ae)
TLS_DHE_RSA_WITH_AES_128_CCM_8 (0xc0a2)
TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_GCM_SHA256 (0xc086)
TLS_ECDHE_RSA_WITH_CAMELLIA_128_GCM_SHA256 (0xc08a)
TLS_DHE_RSA_WITH_CAMELLIA_128_GCM_SHA256 (0xc07c)
TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_CBC_SHA256 (0xc072)
TLS_ECDHE_RSA_WITH_CAMELLIA_128_CBC_SHA256 (0xc076)
TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA256 (0x00be)
TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA (0x0045)
TLS_RSA_WITH_AES_256_GCM_SHA384 (0x009d)
TLS_RSA_WITH_AES_256_CCM (0xc09d)
TLS_RSA_WITH_AES_256_CBC_SHA256 (0x003d)
TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384 (0xc032)
TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 (0xc02a)
TLS_ECDH_RSA_WITH_AES_256_CBC_SHA (0xc00f)
TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02e)
TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 (0xc026)
TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA (0xc005)
TLS_RSA_WITH_AES_256_CCM_8 (0xc0a1)
TLS_RSA_WITH_CAMELLIA_256_GCM_SHA384 (0xc07b)
TLS_RSA_WITH_CAMELLIA_256_CBC_SHA256 (0x00c0)
TLS_RSA_WITH_CAMELLIA_256_CBC_SHA (0x0084)
TLS_ECDH_RSA_WITH_CAMELLIA_256_GCM_SHA384 (0xc08d)
TLS_ECDH_RSA_WITH_CAMELLIA_256_CBC_SHA384 (0xc079)
TLS_ECDH_ECDSA_WITH_CAMELLIA_256_GCM_SHA384 (0xc089)
TLS_ECDH_ECDSA_WITH_CAMELLIA_256_CBC_SHA384 (0xc075)
TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)
TLS_RSA_WITH_AES_128_CCM (0xc09c)
TLS_RSA_WITH_AES_128_CBC_SHA256 (0x003c)
TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256 (0xc031)
TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256 (0xc029)
TLS_ECDH_RSA_WITH_AES_128_CBC_SHA (0xc00e)
TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02d)
TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256 (0xc025)
TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA (0xc004)
TLS_RSA_WITH_AES_128_CCM_8 (0xc0a0)
TLS_RSA_WITH_CAMELLIA_128_GCM_SHA256 (0xc07a)
TLS_RSA_WITH_CAMELLIA_128_CBC_SHA256 (0x00ba)
TLS_RSA_WITH_CAMELLIA_128_CBC_SHA (0x0041)
TLS_ECDH_RSA_WITH_CAMELLIA_128_GCM_SHA256 (0xc08c)
TLS_ECDH_RSA_WITH_CAMELLIA_128_CBC_SHA256 (0xc078)
TLS_ECDH_ECDSA_WITH_CAMELLIA_128_GCM_SHA256 (0xc088)
TLS_ECDH_ECDSA_WITH_CAMELLIA_128_CBC_SHA256 (0xc074)
TLS_EMPTY_RENEGOTIATION_INFO_SCSV (0x00ff)

[SmsAlert]

Final Update:

After lengthy discussions and trace logs we have managed to resolve the matter, these are the issues found:

Cellular module had DNS resolution issues for blynk.cloud, so using ny3.blynk.cloud
uPython ussl on the cellular module had issues in transferring the data with the socket.read() and socket.recv() and was causing data corruption.
Using the “dataCall()” cellular library and socket.socket() was like opening a bit pipe inside a bit pipe.

Cellular side with uPython working great, thanks all for the help

Admin, please close