Data consumption using AT&T and Azure

[John Greene]

We have an MTCDT-LAT1-U Conduit sending data to Azure over AT&T using a custom Python script written for the Conduit’s native MQTT/LoRa broker. A basic LoRa packet with an 8 character payload is a little less than 1kb. Per AT&T’s information, though, the data consumed to send the packet to Azure is >2kb and some sort of response from Azure chews up another >5kb. So, a single packet uses >7kb. Both Microsoft and AT&T are telling us to start with Multitech. Any ideas as to why a single packet consumes so much data?

[Jeff Hatch]

John,

Are you doing encryption. Is this TCP or UDP? What messages are being sent to Azure? Is your connection good? If you have a signal that is weak, TCP will cause retransmissions at a high rate. Until these questions are answered one cannot even speculate. Even UDP will take overhead that will make the 8 LoRa packet characters much bigger. I am suspicious that it is either encryption or weak signal with retransmissions, or both.

Jeff

[John Greene]

Jeff,

We are not doing any encryption. Signal strength appears to be maxed. We are using TCP. Also, the ratios of “raw data” packet size to “sent” and “received” sizes is consistent between tests so I don’t think any retransmission is happening.

John

[Jeff Hatch]

John,

Have you tried tcpdump on ppp0? That might shed some light on what’s going on if the data isn’t encrypted.

Jeff

[John Greene]

Jeff,

Following is the tcpdump on ppp0. Unfortunately, my tcpdump interpretive skills are a bit lacking. While I am researching, any feedback/hints you might be able to give that might help point us in the right direction would be greatly appreciated.

admin@mtcdt:~# tcpdump -i ppp0 -v
tcpdump: listening on ppp0, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
14:16:09.955606 IP (tos 0x0, ttl 64, id 38569, offset 0, flags [DF], proto UDP (17), length 75)
10.53.230.46.13489 > google-public-dns-a.google.com.domain: 41116+ A? VBIOTHUB001.azure-devices.net. (47)
14:16:09.955969 IP (tos 0x0, ttl 64, id 34618, offset 0, flags [DF], proto UDP (17), length 75)
10.53.230.46.13489 > google-public-dns-b.google.com.domain: 41116+ A? VBIOTHUB001.azure-devices.net. (47)
14:16:10.234860 IP (tos 0x58, ttl 56, id 52527, offset 0, flags [none], proto UDP (17), length 131)
google-public-dns-a.google.com.domain > 10.53.230.46.13489: 41116 2/0/0 VBIOTHUB001.azure-devices.net. CNAME ihsu-prod-by-002.cloudapp.net., ihsu-prod-by-002.cloudapp.net. A 104.40.49.44 (103)
14:16:10.235041 IP (tos 0x58, ttl 56, id 23505, offset 0, flags [none], proto UDP (17), length 131)
google-public-dns-b.google.com.domain > 10.53.230.46.13489: 41116 2/0/0 VBIOTHUB001.azure-devices.net. CNAME ihsu-prod-by-002.cloudapp.net., ihsu-prod-by-002.cloudapp.net. A 104.40.49.44 (103)
14:16:10.237000 IP (tos 0x0, ttl 64, id 47542, offset 0, flags [DF], proto TCP (6), length 60)
10.53.230.46.34357 > 104.40.49.44.8883: Flags [S], cksum 0x4579 (correct), seq 2686456706, win 29200, options [mss 1460,sackOK,TS val 109108 ecr 0,nop,wscale 5], length 0
14:16:10.582955 IP (tos 0x0, ttl 64, id 38577, offset 0, flags [DF], proto UDP (17), length 71)
10.53.230.46.63275 > google-public-dns-a.google.com.domain: 5028+ PTR? 46.230.53.10.in-addr.arpa. (43)
14:16:10.707673 IP (tos 0x58, ttl 56, id 3608, offset 0, flags [none], proto UDP (17), length 71)
google-public-dns-a.google.com.domain > 10.53.230.46.63275: 5028 NXDomain 0/0/0 (43)
14:16:10.710373 IP (tos 0x58, ttl 108, id 10000, offset 0, flags [DF], proto TCP (6), length 60)
104.40.49.44.8883 > 10.53.230.46.34357: Flags [S.], cksum 0xb138 (correct), seq 221732293, ack 2686456707, win 8192, options [mss 1380,nop,wscale 8,sackOK,TS val 32995737 ecr 109108], length 0
14:16:10.710746 IP (tos 0x0, ttl 64, id 47543, offset 0, flags [DF], proto TCP (6), length 52)
10.53.230.46.34357 > 104.40.49.44.8883: Flags [.], cksum 0xfbf5 (correct), ack 1, win 913, options [nop,nop,TS val 109155 ecr 32995737], length 0
14:16:10.712627 IP (tos 0x0, ttl 64, id 47544, offset 0, flags [DF], proto TCP (6), length 221)
10.53.230.46.34357 > 104.40.49.44.8883: Flags [P.], cksum 0x15ff (correct), seq 1:170, ack 1, win 913, options [nop,nop,TS val 109155 ecr 32995737], length 169
14:16:10.798944 IP (tos 0x58, ttl 108, id 10001, offset 0, flags [DF], proto TCP (6), length 1420)
104.40.49.44.8883 > 10.53.230.46.34357: Flags [.], cksum 0x12fa (correct), seq 1:1369, ack 170, win 513, options [nop,nop,TS val 32995745 ecr 109155], length 1368
14:16:10.799341 IP (tos 0x0, ttl 64, id 47545, offset 0, flags [DF], proto TCP (6), length 52)
10.53.230.46.34357 > 104.40.49.44.8883: Flags [.], cksum 0xf589 (correct), ack 1369, win 1003, options [nop,nop,TS val 109164 ecr 32995745], length 0
14:16:10.803928 IP (tos 0x58, ttl 108, id 10002, offset 0, flags [DF], proto TCP (6), length 1420)
104.40.49.44.8883 > 10.53.230.46.34357: Flags [.], cksum 0x2f5b (correct), seq 1369:2737, ack 170, win 513, options [nop,nop,TS val 32995745 ecr 109155], length 1368
14:16:10.804236 IP (tos 0x0, ttl 64, id 47546, offset 0, flags [DF], proto TCP (6), length 52)
10.53.230.46.34357 > 104.40.49.44.8883: Flags [.], cksum 0xefd5 (correct), ack 2737, win 1094, options [nop,nop,TS val 109165 ecr 32995745], length 0
14:16:10.804367 IP (tos 0x58, ttl 108, id 10003, offset 0, flags [DF], proto TCP (6), length 895)
104.40.49.44.8883 > 10.53.230.46.34357: Flags [P.], cksum 0x9887 (correct), seq 2737:3580, ack 170, win 513, options [nop,nop,TS val 32995745 ecr 109155], length 843
14:16:10.804715 IP (tos 0x0, ttl 64, id 47547, offset 0, flags [DF], proto TCP (6), length 52)
10.53.230.46.34357 > 104.40.49.44.8883: Flags [.], cksum 0xec35 (correct), ack 3580, win 1179, options [nop,nop,TS val 109165 ecr 32995745], length 0
14:16:10.924826 IP (tos 0x0, ttl 64, id 47548, offset 0, flags [DF], proto TCP (6), length 198)
10.53.230.46.34357 > 104.40.49.44.8883: Flags [P.], cksum 0x77a3 (correct), seq 170:316, ack 3580, win 1179, options [nop,nop,TS val 109177 ecr 32995745], length 146
14:16:10.944570 IP (tos 0x0, ttl 64, id 47549, offset 0, flags [DF], proto TCP (6), length 52)
10.53.230.46.34357 > 104.40.49.44.8883: Flags [F.], cksum 0xeb94 (correct), seq 316, ack 3580, win 1179, options [nop,nop,TS val 109179 ecr 32995745], length 0
14:16:10.980757 IP (tos 0x0, ttl 64, id 61973, offset 0, flags [DF], proto TCP (6), length 60)
10.53.230.46.34358 > 104.40.49.44.8883: Flags [S], cksum 0x1779 (correct), seq 2297190507, win 29200, options [mss 1460,sackOK,TS val 109182 ecr 0,nop,wscale 5], length 0
14:16:11.006680 IP (tos 0x58, ttl 108, id 10004, offset 0, flags [DF], proto TCP (6), length 111)
104.40.49.44.8883 > 10.53.230.46.34357: Flags [P.], cksum 0x11ce (correct), seq 3580:3639, ack 316, win 512, options [nop,nop,TS val 32995767 ecr 109177], length 59
14:16:11.007061 IP (tos 0x58, ttl 64, id 124, offset 0, flags [DF], proto TCP (6), length 40)
10.53.230.46.34357 > 104.40.49.44.8883: Flags [R], cksum 0xc861 (correct), seq 2686457022, win 0, length 0
14:16:11.022686 IP (tos 0x58, ttl 108, id 10005, offset 0, flags [DF], proto TCP (6), length 52)
104.40.49.44.8883 > 10.53.230.46.34357: Flags [.], cksum 0xeddc (correct), ack 317, win 512, options [nop,nop,TS val 32995769 ecr 109179], length 0
14:16:11.023082 IP (tos 0x58, ttl 64, id 125, offset 0, flags [DF], proto TCP (6), length 40)
10.53.230.46.34357 > 104.40.49.44.8883: Flags [R], cksum 0xc860 (correct), seq 2686457023, win 0, length 0
14:16:11.023201 IP (tos 0x58, ttl 108, id 10006, offset 0, flags [DF], proto TCP (6), length 52)
104.40.49.44.8883 > 10.53.230.46.34357: Flags [F.], cksum 0xeddb (correct), seq 3639, ack 317, win 512, options [nop,nop,TS val 32995769 ecr 109179], length 0
14:16:11.023618 IP (tos 0x58, ttl 64, id 126, offset 0, flags [DF], proto TCP (6), length 40)
10.53.230.46.34357 > 104.40.49.44.8883: Flags [R], cksum 0xc860 (correct), seq 2686457023, win 0, length 0
14:16:11.047669 IP (tos 0x58, ttl 108, id 27435, offset 0, flags [DF], proto TCP (6), length 60)
104.40.49.44.8883 > 10.53.230.46.34358: Flags [S.], cksum 0x015a (correct), seq 2305905614, ack 2297190508, win 8192, options [mss 1380,nop,wscale 8,sackOK,TS val 33130802 ecr 109182], length 0
14:16:11.048047 IP (tos 0x0, ttl 64, id 61974, offset 0, flags [DF], proto TCP (6), length 52)
10.53.230.46.34358 > 104.40.49.44.8883: Flags [.], cksum 0x4c3f (correct), ack 1, win 913, options [nop,nop,TS val 109189 ecr 33130802], length 0
14:16:11.049574 IP (tos 0x0, ttl 64, id 61975, offset 0, flags [DF], proto TCP (6), length 221)
10.53.230.46.34358 > 104.40.49.44.8883: Flags [P.], cksum 0xf567 (correct), seq 1:170, ack 1, win 913, options [nop,nop,TS val 109189 ecr 33130802], length 169
14:16:11.130033 IP (tos 0x58, ttl 108, id 27436, offset 0, flags [DF], proto TCP (6), length 1420)
104.40.49.44.8883 > 10.53.230.46.34358: Flags [.], cksum 0x4d62 (correct), seq 1:1369, ack 170, win 513, options [nop,nop,TS val 33130810 ecr 109189], length 1368
14:16:11.130377 IP (tos 0x0, ttl 64, id 61976, offset 0, flags [DF], proto TCP (6), length 52)
10.53.230.46.34358 > 104.40.49.44.8883: Flags [.], cksum 0x45d4 (correct), ack 1369, win 1003, options [nop,nop,TS val 109197 ecr 33130810], length 0
14:16:11.134403 IP (tos 0x58, ttl 108, id 27437, offset 0, flags [DF], proto TCP (6), length 1420)
104.40.49.44.8883 > 10.53.230.46.34358: Flags [.], cksum 0x7fa4 (correct), seq 1369:2737, ack 170, win 513, options [nop,nop,TS val 33130810 ecr 109189], length 1368
14:16:11.134811 IP (tos 0x0, ttl 64, id 61977, offset 0, flags [DF], proto TCP (6), length 52)
10.53.230.46.34358 > 104.40.49.44.8883: Flags [.], cksum 0x4020 (correct), ack 2737, win 1094, options [nop,nop,TS val 109198 ecr 33130810], length 0
14:16:11.134941 IP (tos 0x58, ttl 108, id 27438, offset 0, flags [DF], proto TCP (6), length 895)
104.40.49.44.8883 > 10.53.230.46.34358: Flags [P.], cksum 0x6ffa (correct), seq 2737:3580, ack 170, win 513, options [nop,nop,TS val 33130810 ecr 109189], length 843
14:16:11.135209 IP (tos 0x0, ttl 64, id 61978, offset 0, flags [DF], proto TCP (6), length 52)
10.53.230.46.34358 > 104.40.49.44.8883: Flags [.], cksum 0x3c80 (correct), ack 3580, win 1179, options [nop,nop,TS val 109198 ecr 33130810], length 0
14:16:11.310418 IP (tos 0x0, ttl 64, id 61979, offset 0, flags [DF], proto TCP (6), length 198)
10.53.230.46.34358 > 104.40.49.44.8883: Flags [P.], cksum 0x7374 (correct), seq 170:316, ack 3580, win 1179, options [nop,nop,TS val 109215 ecr 33130810], length 146
14:16:11.397674 IP (tos 0x58, ttl 108, id 27439, offset 0, flags [DF], proto TCP (6), length 111)
104.40.49.44.8883 > 10.53.230.46.34358: Flags [P.], cksum 0xdba0 (correct), seq 3580:3639, ack 316, win 512, options [nop,nop,TS val 33130837 ecr 109215], length 59
14:16:11.397996 IP (tos 0x0, ttl 64, id 61980, offset 0, flags [DF], proto TCP (6), length 52)
10.53.230.46.34358 > 104.40.49.44.8883: Flags [.], cksum 0x3b7e (correct), ack 3639, win 1179, options [nop,nop,TS val 109224 ecr 33130837], length 0
14:16:11.410963 IP (tos 0x0, ttl 64, id 61981, offset 0, flags [DF], proto TCP (6), length 430)
10.53.230.46.34358 > 104.40.49.44.8883: Flags [P.], cksum 0x45f0 (correct), seq 316:694, ack 3639, win 1179, options [nop,nop,TS val 109225 ecr 33130837], length 378
14:16:11.508708 IP (tos 0x58, ttl 108, id 27440, offset 0, flags [DF], proto TCP (6), length 89)
104.40.49.44.8883 > 10.53.230.46.34358: Flags [P.], cksum 0x352f (correct), seq 3639:3676, ack 694, win 510, options [nop,nop,TS val 33130848 ecr 109225], length 37
14:16:11.519449 IP (tos 0x0, ttl 64, id 61982, offset 0, flags [DF], proto TCP (6), length 254)
10.53.230.46.34358 > 104.40.49.44.8883: Flags [P.], cksum 0xd4c3 (correct), seq 694:896, ack 3676, win 1179, options [nop,nop,TS val 109236 ecr 33130848], length 202
14:16:11.594712 IP (tos 0x58, ttl 108, id 27441, offset 0, flags [DF], proto TCP (6), length 89)
104.40.49.44.8883 > 10.53.230.46.34358: Flags [P.], cksum 0x62a9 (correct), seq 3676:3713, ack 896, win 510, options [nop,nop,TS val 33130857 ecr 109236], length 37
14:16:11.595026 IP (tos 0x0, ttl 64, id 61983, offset 0, flags [DF], proto TCP (6), length 712)
10.53.230.46.34358 > 104.40.49.44.8883: Flags [P.], cksum 0xc513 (correct), seq 896:1556, ack 3713, win 1179, options [nop,nop,TS val 109244 ecr 33130857], length 660
14:16:11.687666 IP (tos 0x58, ttl 108, id 27442, offset 0, flags [DF], proto TCP (6), length 89)
104.40.49.44.8883 > 10.53.230.46.34358: Flags [P.], cksum 0xd2be (correct), seq 3713:3750, ack 1556, win 513, options [nop,nop,TS val 33130866 ecr 109244], length 37
14:16:11.687985 IP (tos 0x0, ttl 64, id 61984, offset 0, flags [DF], proto TCP (6), length 158)
10.53.230.46.34358 > 104.40.49.44.8883: Flags [P.], cksum 0xbeda (correct), seq 1556:1662, ack 3750, win 1179, options [nop,nop,TS val 109253 ecr 33130866], length 106
14:16:11.767753 IP (tos 0x58, ttl 108, id 27443, offset 0, flags [DF], proto TCP (6), length 233)
104.40.49.44.8883 > 10.53.230.46.34358: Flags [P.], cksum 0x915a (correct), seq 3750:3931, ack 1662, win 512, options [nop,nop,TS val 33130874 ecr 109253], length 181
14:16:11.790701 IP (tos 0x58, ttl 108, id 27444, offset 0, flags [DF], proto TCP (6), length 137)
104.40.49.44.8883 > 10.53.230.46.34358: Flags [P.], cksum 0x1d82 (correct), seq 3931:4016, ack 1662, win 512, options [nop,nop,TS val 33130877 ecr 109253], length 85
14:16:11.791271 IP (tos 0x0, ttl 64, id 61985, offset 0, flags [DF], proto TCP (6), length 52)
10.53.230.46.34358 > 104.40.49.44.8883: Flags [.], cksum 0x3421 (correct), ack 4016, win 1265, options [nop,nop,TS val 109263 ecr 33130874], length 0
14:16:12.052501 IP (tos 0x0, ttl 64, id 61986, offset 0, flags [DF], proto TCP (6), length 126)
10.53.230.46.34358 > 104.40.49.44.8883: Flags [P.], cksum 0x0459 (correct), seq 1662:1736, ack 4016, win 1265, options [nop,nop,TS val 109289 ecr 33130874], length 74
14:16:12.052830 IP (tos 0x0, ttl 64, id 61987, offset 0, flags [DF], proto TCP (6), length 52)
10.53.230.46.34358 > 104.40.49.44.8883: Flags [F.], cksum 0x33bc (correct), seq 1736, ack 4016, win 1265, options [nop,nop,TS val 109289 ecr 33130874], length 0
14:16:12.316726 IP (tos 0x58, ttl 108, id 27445, offset 0, flags [DF], proto TCP (6), length 52)
104.40.49.44.8883 > 10.53.230.46.34358: Flags [.], cksum 0x3676 (correct), ack 1737, win 512, options [nop,nop,TS val 33130929 ecr 109289], length 0
14:16:12.318208 IP (tos 0x58, ttl 108, id 27446, offset 0, flags [DF], proto TCP (6), length 52)
104.40.49.44.8883 > 10.53.230.46.34358: Flags [F.], cksum 0x3675 (correct), seq 4016, ack 1737, win 512, options [nop,nop,TS val 33130929 ecr 109289], length 0
14:16:12.318641 IP (tos 0x0, ttl 64, id 170, offset 0, flags [DF], proto TCP (6), length 52)
10.53.230.46.34358 > 104.40.49.44.8883: Flags [.], cksum 0x3369 (correct), ack 4017, win 1265, options [nop,nop,TS val 109316 ecr 33130929], length 0
^C
52 packets captured
52 packets received by filter
0 packets dropped by kernel

[Jeff Hatch]

John,

You could try “tcpdump -X -i ppp0”. This output is not showing you what the data is. There are also ways to filter (host ) on ports (port and IPs. The tcpdump on Conduit is the same as on any Linux machine.

Jeff

[Author]

Posts