Why is my IoT device dropping off the MQTT broker ?
November 17, 2020 2:33 PM   Subscribe

This is driving me nuts! :-D I have an IoT device (an ESP32-POE) which is connected to an MQTT broker by ethernet. I specifically chose ethernet because I was fed up with the flakiness of IoT devices on WiFi. Anyway it turns out this IoT device keeps disconnecting from the broker. No obvious pattern to it. Really, really frustrating.

I'm wanting to try to debug where the fault lies, if possible.

I've setup a 'will' topic and am signed up to notifications when the node's will is published by the broker.

I've setup an automation so that when the device reconnects, I get a notification.

What I'm curious about is a nagging feeling that when I'm VNC'ing to the desktop strung off the same network hardware as the ethernet node, the VNC connection frequently fails and I get a socket error then too. I think it's a socket error message. Anyway it struck me that maybe the ethernet cable that feeds the switches down 'that end' of the network may be kinked or faulty, or one of the switches may have an intermittent fault...

Is there any 'ping' or similar function I can setup to really pinpoint WHERE the problem is? Something I can set running for an hour or so and watch out for fails?

Curiously, now, in the evening, that there's nobody AT work, the times when the sensor is 'unavailable' seems to be a lot fewer and further apart...which doesn't really make much sense to me. Yes there are a few more iPhones around during the day on WiFI but the other network traffic is pretty much constant. There's a bit of web browsing not happening in the evening, no card payments being taken on PDQs...but the network IP cameras are still streaming to the NVR, the IP phones are still connected to Asterisk...

And the previous evening this pattern wasn't in evidence either - previous evening while work was empty of people and their iphones the ESP32-POE was toggling available/unavailable just as frequently as it had during the day.

I'm using Esphome so I'm not trying to write all my own connect, keepalive code myself. I'm using a great framework and have really bare bones code. YAML available here if anyone is that interested!

Any help much appreciated...!

Thank you
posted by dance to Computers & Internet (5 answers total) 1 user marked this as a favorite
My suspicion would be the esp32 is disconnecting from the switch, or power cycling.

If you can I would try and log the ethernet status on the esp. depending on the capability of your switch that may be able to log the ethernet port status as well.

You mention not being able to connect to the desktop sometimes, if that is related you could look at / setup logging on that device as well to log the ethernet status. Maybe the switch is acting flaky and taking everything down momentarily.
posted by TheAdamist at 2:50 PM on November 17, 2020

Can you write to a log to track errors over time while removing the variable of being connected? Or is there a serial port you can connect to so you're not doing everything over ethernet?
posted by GuyZero at 3:23 PM on November 17, 2020 [1 favorite]

I'd try two things to start:

1) add some code to broadcast an incrementing integer every 100 mSec, then
2) get Wireshark running somewhere and see if the packets are leaving the node on time. Maybe look to see if DNS requests are failing at the node (or are you hardcoding the broker IP?)
posted by JoeZydeco at 3:30 PM on November 17, 2020 [3 favorites]

What are you running on the ESP32? Micropython makes a great MQTT sender, and it gives you a remote network REPL that you can use to check status and restart if the network.

I didn't know you could use an ESP wired. The wifi is typically solid in my experience, so it may be the power from the POE.
posted by scruss at 4:04 PM on November 17, 2020

@theadamist - thanks, I've installed a brand new switch upstream of the others and have connected JUST the ESP32-POE (and a patch cable to the other switches) to that one to see if that improves things.

@GuyZero there's MQTT logging available I think with ESPHome but otherwise, I think the UART pins overlap with the ethernet pins.

@joeZydeco I'll give the broadcast/wireshark idea a try. Thanks. And yes I'm hardcoding the broker IP.

@scruss running Esphome. You can't use the regular ESP32 wired but you can use the Olimex ESP32-POE wired. & that's why I bought it! :-)
posted by dance at 12:14 AM on November 18, 2020

« Older Petite XL Sweatpants   |   You're still on mute... can you hear me now? Can... Newer »

You are not logged in, either login or create an account to post comments