If a PDF falls in the forest, & no one hears it, does it make a splash?
June 8, 2019 4:37 AM Subscribe
My communications organization doesn't have a robust/consistent way to track publication downloads- now a key indicator in our performance plans (individual and organizational). I want to present ideas, options, and at least an outline of a protocol. Problem: I'm not an IT person.
tl/dr I am an advisor for a communications organization that for *reasons* beyond my control has cut corners on IT. To my surprise, there is no system in place to _consistently_ track downloads of our comms products, though things like 'number of downloads per year' is written into about 30% of the orgs results based management charts. I discovered this when stats on a recent publication of mine were either missing or not registering hits/downloads when news outlets were telling us it was otherwise making a boom. To help get over feelings of 'having worked all for nothing', I want to correct the underlying problem: by suggesting options, tools, and at least a path forward for an otherwise overwhelmed management team (including the comms director) that I believe would welcome the assistance. Wall of technical details follows.
The organization produces both high level policy/science documents and lay person brochures and reporting. These are PDF form (research/policy docs), ebooks (hosted on amazon), infographics/maps/images, website text itself, exposure stories as blog posts, and social media feeds that often link to those exposure stories.
My organization (2 years ago) made a shift over to a results based management format, that is still very unevenly integrated with individual performance reporting processes, as well as document/IT/information management processes. This is an area for management team attention but the specific problem I outline here is urgent and perhaps can illustrate pathways to broader organizational efficiency.
Most staff, including the director have been under the impression that data on the number of downloads of each communication product we produce (both the total, and quantities over time) are being closely monitored. We believe this because there is so much mention of these in results based management forms. Because these indicators were suggested by management, and the director, staff assumed that there would be a protocol at the end of the year to summarize these statistics for the documents we've authored. We also believe this because every 3 months, the director and the board receives data about total number of webhits, geographic location of viewing audience, top searches, and which 3 documents were downloaded the most. This also gave the director and the board the impression that the statistics (and a system to collect/record them over time) were well in hand.
As it turns out, the tracking of these statistics is not organized, strategic, or trackable over time. Apparently, the current process is that the communications team (some are more consistent/proficient than others) create a bit.ly which is then used in updating the organization website (I am not an IT person and I don't know the specifics of this). This allows the communications team to look up the bit.ly address when they have an inquiry (and which has given staff the impression that the information is more organized than it actually is). The bit.ly are not saved in a designated place that other staff can access, staff must ask the comms staff.
Recently, it came to light that we had almost no data on the number of times my major written project had been downloaded. I erroneously assumed my organization was tracking these stats automatically. Our main IT specialist, who also designed the custom website hosting many of my comms products, told me he did not understand the importance of tracking the downloads of the documents, and indicated I should have specifically requested this. (note I include this next info not to place blame but rather to describe the protocol we need to improve) Once he did understand the stats were important, he forgot or inconsistently applied the bit.ly and also posted the materials in so many different places/forms that the tracking information is (as he tells me, too cumbersome to manage). The ebooks are hosted on Amazon, which means that all that comms can look up from them is a single number- we have no way of knowing what that number means (location of the reader, how many pages they viewed etc). The ebook stats are also thus difficult to compare with the pdf stats.
As I understand it from IT staff, organizational reliance on bit.ly may be problematic because 1) the bit.ly's aren't organized/accessible to the authors, 2)they track over time, but as I understand it can only look back so far (?) 3) I'm not sure this is the case, but if staff are copy/pasting from a website addressbar, they may not be picking up the bit.ly when they share it (? I don't know if this is true, or what the criteria would have to be in site construction to make it not true).
At any rate, across the organization, staff/document authors are not informed that they must pay close attention to the links they share (with a protocol) to ensure that the eventual response registers on tracking metrics.
There are 4 ways a reader might get our organizations' documents
1) on the organization's website : Documents and comms products are posted in pdf, ebook, sometimes video, sometimes exposure story blog posts.
2) on a newly created website (Designed in house).
3) on a newly created website (designed outside the organization but my organization gives the comms products to the web desinger)
In cases 2 and 3, these pages are posted under a new domain name, these sites are supposed to help a reader access all the materials related to an initiative, and include links to the same documents that are also hosted on the organization's main site.
4) an organization employee can provide a link to a document (such as a pdf). Currently, the most frequent way an employee does that is to copy and paste the web address of whatever it is they want off the organization website.
I have a good relationship with both the director and comms director and believe my suggestions will be well received (ie. I'm not worried about stepping on toes, and can work with IT staff to solicit their ideas in review before the options are presented to management). But before that, i'd like to prioritize among the list of issues above, pose a range of options to address the most serious, help outline a protocol that all staff can easily/clearly follow, solicit input from IT staff before presenting to management, and then help management monitor whether a new system has rectified the problem over time.
Any suggestions you might have: pathways, tools, subscriptions, process, example protocols that might be working for you etc etc would be greatly appreciated!
tl/dr I am an advisor for a communications organization that for *reasons* beyond my control has cut corners on IT. To my surprise, there is no system in place to _consistently_ track downloads of our comms products, though things like 'number of downloads per year' is written into about 30% of the orgs results based management charts. I discovered this when stats on a recent publication of mine were either missing or not registering hits/downloads when news outlets were telling us it was otherwise making a boom. To help get over feelings of 'having worked all for nothing', I want to correct the underlying problem: by suggesting options, tools, and at least a path forward for an otherwise overwhelmed management team (including the comms director) that I believe would welcome the assistance. Wall of technical details follows.
The organization produces both high level policy/science documents and lay person brochures and reporting. These are PDF form (research/policy docs), ebooks (hosted on amazon), infographics/maps/images, website text itself, exposure stories as blog posts, and social media feeds that often link to those exposure stories.
My organization (2 years ago) made a shift over to a results based management format, that is still very unevenly integrated with individual performance reporting processes, as well as document/IT/information management processes. This is an area for management team attention but the specific problem I outline here is urgent and perhaps can illustrate pathways to broader organizational efficiency.
Most staff, including the director have been under the impression that data on the number of downloads of each communication product we produce (both the total, and quantities over time) are being closely monitored. We believe this because there is so much mention of these in results based management forms. Because these indicators were suggested by management, and the director, staff assumed that there would be a protocol at the end of the year to summarize these statistics for the documents we've authored. We also believe this because every 3 months, the director and the board receives data about total number of webhits, geographic location of viewing audience, top searches, and which 3 documents were downloaded the most. This also gave the director and the board the impression that the statistics (and a system to collect/record them over time) were well in hand.
As it turns out, the tracking of these statistics is not organized, strategic, or trackable over time. Apparently, the current process is that the communications team (some are more consistent/proficient than others) create a bit.ly which is then used in updating the organization website (I am not an IT person and I don't know the specifics of this). This allows the communications team to look up the bit.ly address when they have an inquiry (and which has given staff the impression that the information is more organized than it actually is). The bit.ly are not saved in a designated place that other staff can access, staff must ask the comms staff.
Recently, it came to light that we had almost no data on the number of times my major written project had been downloaded. I erroneously assumed my organization was tracking these stats automatically. Our main IT specialist, who also designed the custom website hosting many of my comms products, told me he did not understand the importance of tracking the downloads of the documents, and indicated I should have specifically requested this. (note I include this next info not to place blame but rather to describe the protocol we need to improve) Once he did understand the stats were important, he forgot or inconsistently applied the bit.ly and also posted the materials in so many different places/forms that the tracking information is (as he tells me, too cumbersome to manage). The ebooks are hosted on Amazon, which means that all that comms can look up from them is a single number- we have no way of knowing what that number means (location of the reader, how many pages they viewed etc). The ebook stats are also thus difficult to compare with the pdf stats.
As I understand it from IT staff, organizational reliance on bit.ly may be problematic because 1) the bit.ly's aren't organized/accessible to the authors, 2)they track over time, but as I understand it can only look back so far (?) 3) I'm not sure this is the case, but if staff are copy/pasting from a website addressbar, they may not be picking up the bit.ly when they share it (? I don't know if this is true, or what the criteria would have to be in site construction to make it not true).
At any rate, across the organization, staff/document authors are not informed that they must pay close attention to the links they share (with a protocol) to ensure that the eventual response registers on tracking metrics.
There are 4 ways a reader might get our organizations' documents
1) on the organization's website : Documents and comms products are posted in pdf, ebook, sometimes video, sometimes exposure story blog posts.
2) on a newly created website (Designed in house).
3) on a newly created website (designed outside the organization but my organization gives the comms products to the web desinger)
In cases 2 and 3, these pages are posted under a new domain name, these sites are supposed to help a reader access all the materials related to an initiative, and include links to the same documents that are also hosted on the organization's main site.
4) an organization employee can provide a link to a document (such as a pdf). Currently, the most frequent way an employee does that is to copy and paste the web address of whatever it is they want off the organization website.
I have a good relationship with both the director and comms director and believe my suggestions will be well received (ie. I'm not worried about stepping on toes, and can work with IT staff to solicit their ideas in review before the options are presented to management). But before that, i'd like to prioritize among the list of issues above, pose a range of options to address the most serious, help outline a protocol that all staff can easily/clearly follow, solicit input from IT staff before presenting to management, and then help management monitor whether a new system has rectified the problem over time.
Any suggestions you might have: pathways, tools, subscriptions, process, example protocols that might be working for you etc etc would be greatly appreciated!
Best answer: The suggestion to use Google Tag Manager is a good one, but it's important to apply it to ALL of your domains. Since you are dealing with multiple domains, make sure to take note of Tag Manager's instructions regarding "cross-domain measurement".
posted by spudsilo at 7:24 AM on June 8, 2019 [1 favorite]
posted by spudsilo at 7:24 AM on June 8, 2019 [1 favorite]
Best answer: Do you know who manages your website or the servers it runs on? All web server software has the ability to log every time it serves a page or file. This logging may already be enabled on your server and if not it's extremely simple for the administrator to enable it.
There is software available that can read these log files and let you do statistical analysis. I just found this one, which starts at $79.
posted by duoshao at 8:10 AM on June 8, 2019 [4 favorites]
There is software available that can read these log files and let you do statistical analysis. I just found this one, which starts at $79.
posted by duoshao at 8:10 AM on June 8, 2019 [4 favorites]
Response by poster: The website and server is maintained by the same IT guy who did not prioritize tracking to date. I can pass on the very good suggestions above, and am more generally wondering if this issue is
a) so automatic/trivial that most organizations do not bother formalizing guidance/protocol as to what to do?
b) something that requires specific training or understanding that the IT guy may not have (he just finished an undergrad degree, I don't know his credentials but I do know the hiring committee lost all 5 of their top 5 IT recruits because they weren't willing to pay them enough to perform the terms of reference. (ie. the google tag manager seems so obvious I can't help but wonder why it hasn't been used to date, or if it only offers a partial solution to the issues I wrote above)
c) would benefit from having a consultant from the outside come in, assess the current situation and establish the protocol that is followed going forward
d) simply a matter of copy/pasting an existing protocol from another organization (if we could get our hands on one), and then getting everyone in comms to agree to stick to it?
posted by iiniisfree at 11:49 AM on June 8, 2019
a) so automatic/trivial that most organizations do not bother formalizing guidance/protocol as to what to do?
b) something that requires specific training or understanding that the IT guy may not have (he just finished an undergrad degree, I don't know his credentials but I do know the hiring committee lost all 5 of their top 5 IT recruits because they weren't willing to pay them enough to perform the terms of reference. (ie. the google tag manager seems so obvious I can't help but wonder why it hasn't been used to date, or if it only offers a partial solution to the issues I wrote above)
c) would benefit from having a consultant from the outside come in, assess the current situation and establish the protocol that is followed going forward
d) simply a matter of copy/pasting an existing protocol from another organization (if we could get our hands on one), and then getting everyone in comms to agree to stick to it?
posted by iiniisfree at 11:49 AM on June 8, 2019
Best answer: a) so automatic/trivial that most organizations do not bother formalizing guidance/protocol as to what to do?
Usually, when there is a business case for some sort of reporting, the collection and generation are automated and reports are emailed/whatever to relevant parties.
You have a case where the business process and the IT process do not line up. This happens a lot, so it doesn't really matter why* - it's (given some assumptions) an easy enough problem to fix**.
Whether the IT dude has the skills and bandwidth to build this is a separate question. It's been my experience that in many places, IT people are dealing with day to day stuff and don't have as much time as necessary for projects like this.
Whatever the case - you need to figure out what data you want, from what sources, how it is to be parsed and presented and to whom, and - this is the key part - who is going to maintain this infrastructure going forward.
* It happens a lot that manager types will get a notion (databases colored blue are all the rage!) present a half-baked idea to the IT peep and then not manage any aspect of the process, leaving the IT peep to guess at what is really expected - and then complain about the result. A skill that takes time to develop is learning which of these notions are worthwhile to spend your time on. It is possible that your IT peep could use better guidance that this is a real priority and not a "wouldn't it be nice" thing.
**You really need to have a clear idea of what data you have, where it is, what you would like done to it, how it should be presented and to whom it should be sent. You cannot count your IT peep to intuit this. You'll have to collaborate with them on this - For example, getting data from your own servers is various degrees of easy, from 3rd parties is less so. There are literally a million ways to do this - you'll have to guide them as to what you would like.
posted by Pogo_Fuzzybutt at 12:43 PM on June 8, 2019
Usually, when there is a business case for some sort of reporting, the collection and generation are automated and reports are emailed/whatever to relevant parties.
You have a case where the business process and the IT process do not line up. This happens a lot, so it doesn't really matter why* - it's (given some assumptions) an easy enough problem to fix**.
Whether the IT dude has the skills and bandwidth to build this is a separate question. It's been my experience that in many places, IT people are dealing with day to day stuff and don't have as much time as necessary for projects like this.
Whatever the case - you need to figure out what data you want, from what sources, how it is to be parsed and presented and to whom, and - this is the key part - who is going to maintain this infrastructure going forward.
* It happens a lot that manager types will get a notion (databases colored blue are all the rage!) present a half-baked idea to the IT peep and then not manage any aspect of the process, leaving the IT peep to guess at what is really expected - and then complain about the result. A skill that takes time to develop is learning which of these notions are worthwhile to spend your time on. It is possible that your IT peep could use better guidance that this is a real priority and not a "wouldn't it be nice" thing.
**You really need to have a clear idea of what data you have, where it is, what you would like done to it, how it should be presented and to whom it should be sent. You cannot count your IT peep to intuit this. You'll have to collaborate with them on this - For example, getting data from your own servers is various degrees of easy, from 3rd parties is less so. There are literally a million ways to do this - you'll have to guide them as to what you would like.
posted by Pogo_Fuzzybutt at 12:43 PM on June 8, 2019
Best answer: In my experience, tracking downloads, site traffic, etc. is important to the marketing department and not the IT department. IT needs to do some things to enable and report on that tracking but they will not do it unless asked, and they will not do it correctly unless given very specific instructions. My IT folks are very experienced and did not do it correctly until I found a marketing consultant who could explain to them how to do it so I can see what I need to see (also cross-domain PDF downloads, for what it's worth). IT is not doing anything wrong in this situation; it's a marketing function that requires another department to do something that either isn't well defined or well communicated. Expecting IT to tell you the "how" isn't going to be helpful; it's not their job.
Google Tag Manager, event tracking and Google Analytics are the tools I use to track this. Tag Manager was designed to get IT out of the process so marketing departments can have more control but it requires a level of IT experience most marketers don't have. Hence in my case, a consultant.
posted by peanut_mcgillicuty at 2:25 PM on June 8, 2019 [1 favorite]
Google Tag Manager, event tracking and Google Analytics are the tools I use to track this. Tag Manager was designed to get IT out of the process so marketing departments can have more control but it requires a level of IT experience most marketers don't have. Hence in my case, a consultant.
posted by peanut_mcgillicuty at 2:25 PM on June 8, 2019 [1 favorite]
Response by poster: Great, you all are helping me narrow it down, thank you. I understand IT isn't doing anything _wrong_, but we don't have a marketing person much less one who has the IT experience you all are describing. In a pinch, I'm trying to bridge the gap. I think what I'm understanding from the answers above is my organization could benefit from bringing in a marketing consultant who could explain to IT how to enable and report on the tracking (and in particular, create a protocol that will ensure the tracking includes cross-domain PDF downloads). Does that description capture all the needs? If so, its valuable text for us to have, not only for the Terms of Reference for a consultant, but also in a wishlist of skills should we make another long-term hire.
posted by iiniisfree at 8:57 PM on June 8, 2019
posted by iiniisfree at 8:57 PM on June 8, 2019
Best answer: If you're looking for a wishlist of skills for a consultant or future hire, you might look first at job descriptions for Business Intelligence Analysts or Developers. These are the kinds of roles that are generally responsible for bridging the gap between what you already know about your site usage and what you want to know.
posted by smuna at 8:09 AM on June 9, 2019
posted by smuna at 8:09 AM on June 9, 2019
This thread is closed to new comments.
See the Google documentation, here: Tag Manager
posted by smuna at 6:51 AM on June 8, 2019 [4 favorites]