Python App Emulator

jaybee73 · June 2, 2021, 8:26am

Hello all
This post may be mute following the announcement of Blynkv2, however

I’ve seen a number of requests for code to emulate the mobile phone app, particularly as the client.jar file is both unavailable and doesn’t really work (at least when I’ve tried it)

However, I recently had a need to develop some code to monitor a Blynk app/dashboard and wanted to try again

After some vigorous examination of the Java source for the server, I modified the blynklib.py file within the Blynk Python library to connect to the server “as the mobile app”

This library is not complete, but does allow logging on to the server, downloading app/dashboard information about the widgets, registering to the dashboard to receive push updates from Blynk, and sending data to virtual pins

Theoretically it could be extended to create projects/dashboards and request new tokens but that is not what I needed to do. I only needed to be able to get events from my dash and respond to them dynamically

You can download the code at the link below. Be safe and unzip and read the code before trying to run it

Download Code

Jason

jaybee73 · June 11, 2021, 1:56pm

An updated version of the code above is now available at the same link, I’ve:

Made the whole library event driven
Put back in the heartbeat functionality (as the Blynk server seems to kill the connection after five minutes)
Re-organised the code to make it easier to modify/add new features
Added comments everywhere so you can see how it all works
Improved/simplified the logging

Other than that, the demo does not much more than before but should be easier to work on for anyone wanting to use it

Again, I realise that the old Blynk is going (eventually) but I think this re-organisation can be ported back to the main blynklib library (for edge devices) and should make it easier to maintain going forward. Feel free to use the code for that purpose

The main re-factorisation is to completely remove the message construction/parsing out of the Blynk class and into its own module, this makes both sets of code much cleaner and easier to read

Jason

Gunner · June 12, 2021, 7:03pm

Nice…

And another opportunity to increase my Python experience, albeit like jumping from kindergarten to collage

I think I have hardcoded the connection info correctly. A bit unsure of the certificate, do I need to provide that file somehow?

However, I do get an error that is stumping me at the moment. What file or directory is it expecting?

2021-06-12 13:04:48,922 [DEBUG]  Request(server details)
2021-06-12 13:04:48,922 [INFO]  Connecting to <REDACTED>.ddns.net:9443
2021-06-12 13:05:00,077 [INFO]  Using SSL socket...
2021-06-12 13:05:00,077 [ERROR]  Connection with the Blynk server failed: [Errno 2] No such file or directory
2021-06-12 13:05:00,077 [WARNING]  Unhandled Event(disconnect) -> ()
2021-06-12 13:05:01,234 [INFO]  Registered request handlers: ['server details', 'auth details']
2021-06-12 13:05:01,234 [INFO]  Registered event handlers: ['connect', 'load profile', 'write v0', 'write v1', 'write v2', 'write v3', 'write v4', 'write v5', 'write v6', 'write v7', 'write v8', 'write v9', 'write v10', 'write v11', 'write v12', 'write v13', 'write v14', 'write v15', 'write v16', 'write v17', 'write v18', 'write v19', 'write v20', 'write v21', 'write v22', 'write v23', 'write v24', 'write v25', 'write v26', 'write v27', 'write v28', 'write v29', 'write v30', 'write v31', 'write v32']

Gunner · June 12, 2021, 8:24pm

I think I found where to hardcode my project name (I am guessing it doesn’t look for all availed and ask the user).

blynkapp = {'dashname': 'zeRGBa - RGB LED', 'widgets': {'button': {}, 'sensor': {}}}

I also tried hard spaces…

blynkapp = {'dashname': 'zeRGBa_-_RGB_LED', 'widgets': {'button': {}, 'sensor': {}}}

And I seem to get better “results” when switching 'ssl': False But still not clear if something more is missing on my part, or some other issue with the blynktimer.py code?

        ___  __          __  
       / _ )/ /_ _____  / /__
      / _  / / // / _ \/  '_/
     /____/_/\_, /_//_/_/\_\ 
       / _ |/___/ ___  for      
      / __ |/ _ \/ _ \ Python      
     /_/ |_/ .__/ .__/ v0.2.6_app      
          /_/  /_/

2021-06-12 13:17:52,001 [INFO]  Registered request handlers: ['server details', 'auth details']
2021-06-12 13:17:52,001 [INFO]  Registered event handlers: ['connect', 'load profile', 'write v0', 'write v1', 'write v2', 'write v3', 'write v4', 'write v5', 'write v6', 'write v7', 'write v8', 'write v9', 'write v10', 'write v11', 'write v12', 'write v13', 'write v14', 'write v15', 'write v16', 'write v17', 'write v18', 'write v19', 'write v20', 'write v21', 'write v22', 'write v23', 'write v24', 'write v25', 'write v26', 'write v27', 'write v28', 'write v29', 'write v30', 'write v31', 'write v32']
2021-06-12 13:17:52,001 [DEBUG]  Request(server details)
2021-06-12 13:17:52,016 [INFO]  Connecting to <REDACTED>.ddns.net:9443
2021-06-12 13:18:04,131 [INFO]  Connection to Blynk server established
2021-06-12 13:18:04,147 [DEBUG]  Request(auth details)
2021-06-12 13:18:04,162 [INFO]  Authenticating user: <REDACTED>@gmail.com
2021-06-12 13:18:04,178 [DEBUG]  Sending BlynkLoginMessage - BlynkCommand(Login) ID(1): ['<REDACTED>@gmail.com', 'VyD3Yy0nyzln<REDACTED IN PART>amalM6WKjxU=', 'Python App', '0.2.6_app', 'Blynk']

Then a long pause before this…

Traceback (most recent call last):
  File "C:\Users\Gunner\blynk_app_demo\blynk_app.py", line 215, in <module>
    main()
  File "C:\Users\Gunner\blynk_app_demo\blynk_app.py", line 212, in main
    timer.run()
  File "C:\Users\Gunner\blynk_app_demo\blynktimer.py", line 102, in run
    timers_intervals = [curr_timer.run() for curr_timer in Timer.timers.values() if not curr_timer.stopped]
  File "C:\Users\Gunner\blynk_app_demo\blynktimer.py", line 102, in <listcomp>
    timers_intervals = [curr_timer.run() for curr_timer in Timer.timers.values() if not curr_timer.stopped]
  File "C:\Users\Gunner\blynk_app_demo\blynktimer.py", line 127, in run
    self.deco(*self.args, **self.kwargs)
  File "C:\Users\Gunner\blynk_app_demo\blynk_app.py", line 198, in toggle_door_button
    blynkapp['widgets']['button']['value'] = int(blynkapp['widgets']['button']['value']) + 1
KeyError: 'value'

jaybee73 · June 13, 2021, 6:28am

Gunner

It looks like you have correctly encoded the information

The ./cert_chain.cert needs to be a local file on your computer which contains your local Blynk Server SSL certificate for verifying the server during connection. You have two options:

You are using a locally signed certificate at the server, you should copy the public certificate to your computer and provide the filename in the code
Your server certificate has been signed by an external certificate authority. In this case you should remove the certificate key from the code (ie. return { 'server: ‘xxxx.yy.net’, ‘port’:9443, ‘ssl’:True} )

The program here is failing because it cannot find the certificate file to load

Jason

jaybee73 · June 13, 2021, 6:30am

YOu really need to use SSL as True (not sure why I made it optional other than to keep it partially the same as the pre-existing blynklib library)

When connecting to the harware interface, both SSL and plaintext is OK, when connecting to the app port, SSL is required

Jason

jaybee73 · June 13, 2021, 6:43am

I will also add, the pause is due to the way the demo is coded. After the timer expires, it tries to push the “widget” button and if you are not connected then everything falls in a hole

The demo app is written to assume that the connection will succeed (including authentication)

Gunner · June 13, 2021, 6:57pm

OK, thanks… I was suspicious of that. My usual RPi with Blynk Server is packed away, so I had setup a Windows 10, Docker based, Local Server. But with the free version of Docker, I don’t seem to have direct file access, and haven’t figured out how to build/compile everything before installing it to Docker.

I will test the 2nd option first… Nope… it’s not liking that.

I will either dig out my RPi, or learn me some pre-Docker prep.

Thanks.

Gunner · June 13, 2021, 8:55pm

Dig out the RPi it is… It is time to play with other Apps and it anyhow.

It has been awhile, but I think I had generated my own certs… ended up with this in my server.properties

server.ssl.cert=/home/pi/Blynk/server.crt
server.ssl.key=/home/pi/Blynk/server.pem
server.ssl.key.pass=<redacted>

So I tried this in your demo…

return {'server': '10.10.3.13', 'port': 9443, 'ssl': True, 'certificate': '/home/pi/Blynk/server.crt'}

…and this with the dot in ./ …

return {'server': '10.10.3.13', 'port': 9443, 'ssl': True, 'certificate': './home/pi/Blynk/server.crt'}

But still getting path errors. Does it need the .pem and key.pass as well?

2021-06-13 13:43:51,951 [DEBUG]  Request(server details)
2021-06-13 13:43:51,956 [INFO]  Connecting to 10.10.3.13:9443
2021-06-13 13:43:51,984 [INFO]  Using SSL socket...
2021-06-13 13:43:51,997 [ERROR]  Connection with the Blynk server failed: [Errno 2] No such file or directory

Not sure if I am hitting “Py Noob” issues or just not understanding Linux path structures… or your demo

Gunner · June 13, 2021, 11:08pm

I decided to just try the cloud server… same…

return {'server': 'blynk-cloud.com', 'port': 9443, 'ssl': True, 'certificate': './cert_chain.crt'}

2021-06-13 16:05:12,079 [DEBUG]  Request(server details)
2021-06-13 16:05:12,079 [INFO]  Connecting to blynk-cloud.com:9443
2021-06-13 16:05:12,219 [INFO]  Using SSL socket...
2021-06-13 16:05:12,219 [ERROR]  Connection with the Blynk server failed: [Errno 2] No such file or directory

I, as is the cert file, appear to be lost

jaybee73 · June 14, 2021, 2:52am

Now we are making some progress, excellent

The server.crt file is the one you want, but it needs to be copied to the computer you are running the Python program on. Do not copy the pem file, that is private for the Blynk server, the crt file is the one used by clients to initiate the connection.

Once you copy it to your desktop/laptop where you are working from, you now need to specify the path in:

return {‘server’: ‘10.10.3.13’, ‘port’: 9443, ‘ssl’: True, ‘certificate’: ‘/home/pi/Blynk/server.crt’}

How you do this depends on whether you are running Linux or WIndows or Mac on your laptop/desktop. I haven’t run a Windows machine in 15 years and when I use one it is not for programming. The quickest solution would be to put the crt file in the same directory/folder as your program, then in Linux you can use ‘./server.crt’ and in WIndows you can try ‘/server.crt’

The no such file or directory error message is referring to the fact that Python cannot find your server.crt file

With regards to path, if the filename starts with a /, then it will specify a file from the directory root, so the full directory starting from the bottom level to the file. If it starts with any other character, then it specifies the path to the file from the current directory. “./” literally means the current directory. Similarly ./home/pi/Blynk/server.crt means start in the current directory, then search into subdirector home, then subdirectory pi, etc… While ‘/home/pi/Blynk/server.crt’ means that this exactt directory should be on your laptop/desktop. If you are using WIndows I don’t know how to manage it properly as Windows uses both a disk name (eg c:) and the path separators are usually the other way around \ vs /

jaybee73 · June 14, 2021, 2:56am

I am obviously on the other side of the world to you, its a slow 24 hour debug cycle

When connecting to the Blynk cloud server you:

Do NOT need to supply a certificate file, the Blynk cloud server will most definitely have been signed by an approved certificate authority
Most likely should be using port 443 rather than 9443

Finally, a note on your own self-signed certificate. As I run a number of services, instead of going with simple self-signed certificates, I ended up making my own certificate authority and then signing individual certificates with my authority. This enables me to install my certificate authority certificate on end host and they automatically will allow any new services I create without having to copy all the certs around. For this program, I had to concacatenate the Blynk server certificate and the CA Certificate together to pass to the Python program. However, if you just went with a simple self-signed certificate, then you will just need to copy the server.crt file as is

Gunner · June 14, 2021, 6:56am

Well, the errors make slightly more sense… as in syntax (as in actually pointing to the area of interest) if not in logic… the server is on 10.10.3.13 (The RPi on same internal network as the Windows PC which is 10.10.3.11, if that matters) and the cert file was copied from there as instructed, why would it not be valid?

I placed the copy of the server.crt into the same folder as your demo files, thus (by my understanding) this should be correct… (Running on Win10 with Windows Subsystem for Linux installed)

return {'server': '10.10.3.13', 'port': 9443, 'ssl': True, 'certificate': './server.crt'}

… which gets this result…

2021-06-13 23:29:32,402 [DEBUG]  Request(server details)
2021-06-13 23:29:32,403 [INFO]  Connecting to 10.10.3.13:9443
2021-06-13 23:29:32,417 [INFO]  Using SSL socket...
2021-06-13 23:29:32,718 [ERROR]  Connection with the Blynk server failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for '10.10.3.13'. (_ssl.c:1129)

And just to compare, I tested again on Blynk Cloud with port 443 and no reference to a cert file…

return {'server': 'blynk-cloud.com', 'port': 443, 'ssl': True}

2021-06-13 23:46:56,704 [DEBUG]  Request(server details)
2021-06-13 23:46:56,719 [INFO]  Connecting to blynk-cloud.com:443
2021-06-13 23:47:08,910 [INFO]  Using SSL socket...
2021-06-13 23:47:09,145 [ERROR]  Connection with the Blynk server failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1129)

I am wondering if my Blynk Server has been running without proper certification all this time? Is that even possible? I must look into how to confirm one way or other. But that doesn’t explain the Blynk Cloud issue does it.

Gunner · June 14, 2021, 7:11am

Hmmm… Blynk Log says this…

01:34:01.846 INFO - Didn't find Let's Encrypt certificates.
01:34:01.847 INFO - Automatic certificate generation is turned ON.

So…not using my self generated server.crt from way back when?

jaybee73 · June 14, 2021, 7:17am

As to the blynk-cloud server that looks very strange. I can’t check as I don’t have an account, I only run a local server.

As for your local server, did you try going back to the DNS name .ddns.net ??

The code that bonds the SSL connection is on lines 247-255 of blynklib_app.py

if server_details['ssl']:
    self.log.info('Using SSL socket...')
    # If certificate not provided set to None to use system default CA certificates
    if 'certificate' not in server_details:
        server_details['certificate'] = None
    ssl_context = ssl.create_default_context(cafile=server_details['certificate'])
    ssl_context.verify_mode = ssl.CERT_REQUIRED
    self.__socket.settimeout(self.SOCK_SSL_TIMEOUT)
    self.__socket = ssl_context.wrap_socket(sock=self.__socket,
                             server_hostname=server_details['server'])

In essence, this code runs when you set ssl to True
It then creates the ssl context for connection. If the cafile parameter is None, it will juse the system default CA certificates which should match the blynk-cloud ones. If a file is specified, it will load that as a valid CA

You can change the line that sets ssl_context.verify_mode to ssl.CERT_NONE to ignore errors in the certificate just to see if you can get to the next step, I did this when I started until I figured out how to get the certificate working.

Gunner · June 14, 2021, 7:19am

I will try that now… as it is a recent and basically stock setup.

jaybee73 · June 14, 2021, 7:21am

Referring to an earlier post and your server config…

server.ssl.cert=/home/pi/Blynk/server.crt
server.ssl.key=/home/pi/Blynk/server.pem
server.ssl.key.pass=<redacted>

I did NOT use the sever.ssl.key.pass parameter at all, because my certificate did not use a password

However I DID specify the server.host and restore.host and the DNS name of my server

jaybee73 · June 14, 2021, 7:24am

For the blynk-cloud server, you are right it does not appear to be externally signed

If you use your browser and go to https://blynk-cloud.com

Then click on the security bar to view the certificate and export it, it will create a local copy of the certificate file you can use in the connection attempt

Gunner · June 14, 2021, 7:26am

My, not local, local server… same same…

2021-06-14 00:24:40,701 [ERROR] Connection with the Blynk server failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1129)

jaybee73 · June 14, 2021, 7:31am

If you run the following command (Linux) it will give you the certificate information on the file

openssl x509 -in <cert_file_name> -noout -text

The information at Issue 34440: Certificate verify failed (works fine in 3.6) - Python tracker indicates why some certificates can cause problems if they were not created properly

I would try with with the verify mode set to ssl.CERT_NONE to make sure it works first, then worry about the certificate