Did you get that thing I sent you? Google analytics and email tracking
October 18, 2018 7:20 PM Subscribe
I'm working on a project to roll-my-own google script-powered Yet-Another-Mail-Merge. I got it working! Except for the thing they do to track email opens, bounces, etc.
I've never done anything like this before. So I did some googling and signed up for Google Analytics, found a bunch of tutorials about how to do email tracking that all say to do the same thing, and inserted a pseudo image pixel with the tracking ID into my email. That is making its way into my emails but not coming out on the other side. In gmail's web app*, I don't see it when I view source (I only see the img tag), and the times I open my own emails are not showing up in my Google Analytics dashboard. Also, when I view the email in AirMail, I get a little icon saying there should be an image here.
I have some questions
1- How does YAMM do it? Is this proprietary info? There's nothing specific enough on their site.
2- Is it possible to do email tracking using my own image instead of the fake image thing?
3- In addition, I am very far away from figuring out how YAMM updates my spreadsheet with the status of each email. I am inserting my ID of the person getting my email, so if I had the data I'd be able to match it up, but I can't figure out where the data is, even just to view it.
4- If I also want to track clicks on a link I'm sending, do I need to make another property ID in Analytics?
*Google provides technology to track email opening but thinks their users are too good for that?
I've never done anything like this before. So I did some googling and signed up for Google Analytics, found a bunch of tutorials about how to do email tracking that all say to do the same thing, and inserted a pseudo image pixel with the tracking ID into my email. That is making its way into my emails but not coming out on the other side. In gmail's web app*, I don't see it when I view source (I only see the img tag), and the times I open my own emails are not showing up in my Google Analytics dashboard. Also, when I view the email in AirMail, I get a little icon saying there should be an image here.
I have some questions
1- How does YAMM do it? Is this proprietary info? There's nothing specific enough on their site.
2- Is it possible to do email tracking using my own image instead of the fake image thing?
3- In addition, I am very far away from figuring out how YAMM updates my spreadsheet with the status of each email. I am inserting my ID of the person getting my email, so if I had the data I'd be able to match it up, but I can't figure out where the data is, even just to view it.
4- If I also want to track clicks on a link I'm sending, do I need to make another property ID in Analytics?
*Google provides technology to track email opening but thinks their users are too good for that?
Best answer: I understand that, generally speaking, an Ask shouldn't be answered with "don't do that," but this kind of tracking is invasive and unwelcome. The only reason there's not more outcry about it is because most people don't understand what's involved.
People like (and flabdablet and others) consistently advise our friends and family to leave on or turn on the content blocking in email tools to prevent these things from working.
tl;dr: Please don't be part of the privacy-invading problem
posted by uberchet at 6:33 AM on October 19, 2018 [3 favorites]
People like (and flabdablet and others) consistently advise our friends and family to leave on or turn on the content blocking in email tools to prevent these things from working.
tl;dr: Please don't be part of the privacy-invading problem
posted by uberchet at 6:33 AM on October 19, 2018 [3 favorites]
Response by poster: I appreciate the heads up and I don’t want to do anything invasive or violate anyone’s privacy. I didn’t realize this kind of tracking was considered bad I guess because YAMM makes it look so easy. Consider that part of my plan moot. I’m still interested in the answers to my questions from an academic point of view if anyone has them.
posted by bleep at 7:58 AM on October 19, 2018
posted by bleep at 7:58 AM on October 19, 2018
Best answer: 1- How does YAMM do it?
Exactly as badly and hit-or-miss as you'd expect once you understand how email tracking works.
2- Is it possible to do email tracking using my own image instead of the fake image thing?
Yes. All you need to make image-based tracking work is an HTML email containing an <img> tag for which the URL in the src= attribute includes a tracking value unique to that particular email; the usual way to do this is to include a nonce in the URL's query string. The tracking mojo is in the sender-side knowledge of which nonce matches which recipient, not the details of the image.
This stuff is conventionally done using 1x1 transparent web bug images to allow separation of function between tracking servers and mere static image servers, but there's nothing inherent in the method that means that's the only way to do it.
3- In addition, I am very far away from figuring out how YAMM updates my spreadsheet with the status of each email. I am inserting my ID of the person getting my email, so if I had the data I'd be able to match it up, but I can't figure out where the data is, even just to view it.
Once you have some facility that can send emails customized individually with tracking nonces, the access logs for the web server that hosts the tracking images and/or linked web pages will reveal the time each was fetched, the IP address it was fetched from, and the ID of the email it was embedded in. If the handler for the tracking item URL is dynamic, you don't even need to scrape the access logs since all of that information can get pushed into a database automatically as the server builds up the HTTP(S) response.
4- If I also want to track clicks on a link I'm sending, do I need to make another property ID in Analytics?
I'm not familiar enough with Analytics to say for sure, but you might find it more convenient to assign every <img> tag and every <a> tag its own nonce, especially if you're trying to work out stuff like whether you get better click-through rates from links near the start of a spiel as opposed to repeats of the same links at the end. If each link in any given email is unique within that email even before nonces get tacked on, though, you might prefer to re-use the same nonce on all of them - i.e. treat it as an email ID rather than a link ID.
But as others have pointed out above, all this stuff is Dark Arts and best eschewed by any person with an ounce of self-respect.
posted by flabdablet at 10:46 AM on October 19, 2018 [2 favorites]
Exactly as badly and hit-or-miss as you'd expect once you understand how email tracking works.
2- Is it possible to do email tracking using my own image instead of the fake image thing?
Yes. All you need to make image-based tracking work is an HTML email containing an <img> tag for which the URL in the src= attribute includes a tracking value unique to that particular email; the usual way to do this is to include a nonce in the URL's query string. The tracking mojo is in the sender-side knowledge of which nonce matches which recipient, not the details of the image.
This stuff is conventionally done using 1x1 transparent web bug images to allow separation of function between tracking servers and mere static image servers, but there's nothing inherent in the method that means that's the only way to do it.
3- In addition, I am very far away from figuring out how YAMM updates my spreadsheet with the status of each email. I am inserting my ID of the person getting my email, so if I had the data I'd be able to match it up, but I can't figure out where the data is, even just to view it.
Once you have some facility that can send emails customized individually with tracking nonces, the access logs for the web server that hosts the tracking images and/or linked web pages will reveal the time each was fetched, the IP address it was fetched from, and the ID of the email it was embedded in. If the handler for the tracking item URL is dynamic, you don't even need to scrape the access logs since all of that information can get pushed into a database automatically as the server builds up the HTTP(S) response.
4- If I also want to track clicks on a link I'm sending, do I need to make another property ID in Analytics?
I'm not familiar enough with Analytics to say for sure, but you might find it more convenient to assign every <img> tag and every <a> tag its own nonce, especially if you're trying to work out stuff like whether you get better click-through rates from links near the start of a spiel as opposed to repeats of the same links at the end. If each link in any given email is unique within that email even before nonces get tacked on, though, you might prefer to re-use the same nonce on all of them - i.e. treat it as an email ID rather than a link ID.
But as others have pointed out above, all this stuff is Dark Arts and best eschewed by any person with an ounce of self-respect.
posted by flabdablet at 10:46 AM on October 19, 2018 [2 favorites]
Response by poster: Do you think ethically it makes a difference if I’m doing this for a personal project people signed up for, and I’m tracking to better understand and improve their experience, and not to find better ways of selling to them? I can see arguments for both sides.
posted by bleep at 11:17 AM on October 19, 2018
posted by bleep at 11:17 AM on October 19, 2018
If your normal process involves sending bulk emails with links in them then all you're doing is helping train your recipients to fall for phishing scams and malware attacks. Don't Click On Links In Emails is a pretty good bright-line security rule, and expecting people not to follow it is essentially just rudeness.
If you're looking for some kind of quality feedback, I'd strongly advise you to implement some kind of login system that your recipients could and should use completely independent of any emails you send them, along with a Metatalk-alike for discussion-based feedback and a contact form for individual feedback, and avoid putting any tracking stuff at all into the emails themselves - especially if your tracking is based on third-party services that act as tracking intermediaries between the links in your emails and your final URLs.
Personally I block as many advertising and tracking servers as I possibly can at my house's Internet gateway, and I also run uBlock Origin in the browsers I use for webmail and inside my Thunderbird email client as well; that means that all those intermediary-based links are completely broken for me from the get-go, which suits me fine because frankly, the entire advertising industry and all its associated hangers-on can and should go fuck themselves.
If you're seriously interested in improving my experience, make it easy for me to tell you explicitly if and how the present experience requires improving. Don't pester me with surveys or try to second-guess me with metrics and psychology, because that will only irritate me and make me think less of you. I'm sure I'm not alone in this.
posted by flabdablet at 11:37 AM on October 19, 2018 [3 favorites]
If you're looking for some kind of quality feedback, I'd strongly advise you to implement some kind of login system that your recipients could and should use completely independent of any emails you send them, along with a Metatalk-alike for discussion-based feedback and a contact form for individual feedback, and avoid putting any tracking stuff at all into the emails themselves - especially if your tracking is based on third-party services that act as tracking intermediaries between the links in your emails and your final URLs.
Personally I block as many advertising and tracking servers as I possibly can at my house's Internet gateway, and I also run uBlock Origin in the browsers I use for webmail and inside my Thunderbird email client as well; that means that all those intermediary-based links are completely broken for me from the get-go, which suits me fine because frankly, the entire advertising industry and all its associated hangers-on can and should go fuck themselves.
If you're seriously interested in improving my experience, make it easy for me to tell you explicitly if and how the present experience requires improving. Don't pester me with surveys or try to second-guess me with metrics and psychology, because that will only irritate me and make me think less of you. I'm sure I'm not alone in this.
posted by flabdablet at 11:37 AM on October 19, 2018 [3 favorites]
Response by poster: OK well this has been very helpful as this was super hard to get going so I'm happy to not do it as well as happy not to cause annoyance or hassle. Thank you!
posted by bleep at 3:04 PM on October 19, 2018
posted by bleep at 3:04 PM on October 19, 2018
« Older let's talk about chronic couch sleeping. what do? | Best work-safe live webcams/live streams 2018 Newer »
This thread is closed to new comments.
In my experience, most people either don't notice or don't understand the notification that gets displayed when this facility blocks remotely sourced images, and treat emails containing the little missing-image placeholder marks as just Something Email Does.
It's been a long time since I used Gmail's irritatingly unstable web interface but it looks like the current version still has this baked in, albeit turned on and off at random according to some inscrutable internal "safety" heuristic.
posted by flabdablet at 11:18 PM on October 18, 2018 [3 favorites]