Does this piece of software exist? Email it two emails and it'll combine
August 20, 2021 5:39 AM Subscribe
Due to a limitation in some commercial software I use, I'm trying to find a (most likely and probably best) unix tool which would accept two incoming emails for the same recipient (within say a five minute period) and essentially merge them and re send out?
I have a postfix server at work which forwards onto gmail and this works beautifully, apparently there is some (to me) super complicated potential solution involving milters but this goes riiight over my head.
I was thinking there must be a way - I just lack the google-fu to sort it.
Any ideas? And sadly, no, the deficiency in the original software can't be resolved - already tried that!
Thanks
TLDR: How to turn two emails into one where the recipient is the same and the original emails were sent within the same five minute window.**
** Could also match on phrases if the time limit is the thorniest issue.
I have a postfix server at work which forwards onto gmail and this works beautifully, apparently there is some (to me) super complicated potential solution involving milters but this goes riiight over my head.
I was thinking there must be a way - I just lack the google-fu to sort it.
Any ideas? And sadly, no, the deficiency in the original software can't be resolved - already tried that!
Thanks
TLDR: How to turn two emails into one where the recipient is the same and the original emails were sent within the same five minute window.**
** Could also match on phrases if the time limit is the thorniest issue.
This could be done in Power Automate using the Gmail connector to get the contents of both mails, combine them into one mail, and then send that mail out. The trigger could either be the action of the email appearing in Gmail, or if the emails come on a regular frequency you could run the workflow at a set time each day/week/month depending on the frequency.
But you may not want to use a Microsoft solution, although it is web-based and your O/S doesn't matter.
posted by underclocked at 6:59 AM on August 20, 2021 [1 favorite]
But you may not want to use a Microsoft solution, although it is web-based and your O/S doesn't matter.
posted by underclocked at 6:59 AM on August 20, 2021 [1 favorite]
Convoluted idea: Using open-source mailing list software, create a mailing list with just yourself as a subscriber. Forward the mail there. Modify the 'daily digest' feature of the software to be an 'every five minutes' (or timeframe you wish) digest that gets sent to you on that schedule.
It won't combine based on subject of the email without further work, but it will combine email to you every 'n' minutes.
posted by TimHare at 7:24 AM on August 20, 2021 [2 favorites]
It won't combine based on subject of the email without further work, but it will combine email to you every 'n' minutes.
posted by TimHare at 7:24 AM on August 20, 2021 [2 favorites]
It sounds like you mean Outgoing Mail. I was initially thinking of incoming mail.
Do you send any real mail that you want instantly sent all normal like?
Just how does 'commercial software' send it's mail? Does postfix listen on localhost and gets the email via SMTP, or does the software use 'sendmail' or such?
Can you change how/where software sends its email?
If you can change the method/destination that the software sends its emails then this is doable.
You end up with a program or two (depending) that receives the email that software sends, stores it, and throttles or expedites it.
I've done this in both directions. But the sending part was "I'm the software writer" so it was just built-in. If you can get software to either send to a custom SMTP or point it to a different 'sendmail' (the to major methods programs use to send mail) then it's possible.
The main question is HOW does your commercial software send its mail and can you configure that? If you can, your goal is possible.
Are you really sure that your software is actually using your postfix server to send its mail? Maybe it's just using SMTP based on the MX record of the recipient(s) email address(es) and you can't do anything (well I'm sure you COULD use the firewall rules to redirect all SMTP traffic to where you wanted it).
On preview.... I thought 'milter' was a typo for 'filter'. I would craft something out of Perl, some modules, maybe 'procmail'. Actually sending the final email is trivial, the core new thing that does the magic aggregation part is as simple or as complicated as needed, the NEEDS INFORMATION part is the commercial software configuration for its idea of sending email, the fiddly bit is dealing with that to get the emails commercial software sends into the new thing. The rest is a Simple Matter Of Programming (SMOP).
posted by zengargoyle at 7:28 AM on August 20, 2021
Do you send any real mail that you want instantly sent all normal like?
Just how does 'commercial software' send it's mail? Does postfix listen on localhost and gets the email via SMTP, or does the software use 'sendmail' or such?
Can you change how/where software sends its email?
If you can change the method/destination that the software sends its emails then this is doable.
You end up with a program or two (depending) that receives the email that software sends, stores it, and throttles or expedites it.
I've done this in both directions. But the sending part was "I'm the software writer" so it was just built-in. If you can get software to either send to a custom SMTP or point it to a different 'sendmail' (the to major methods programs use to send mail) then it's possible.
The main question is HOW does your commercial software send its mail and can you configure that? If you can, your goal is possible.
Are you really sure that your software is actually using your postfix server to send its mail? Maybe it's just using SMTP based on the MX record of the recipient(s) email address(es) and you can't do anything (well I'm sure you COULD use the firewall rules to redirect all SMTP traffic to where you wanted it).
On preview.... I thought 'milter' was a typo for 'filter'. I would craft something out of Perl, some modules, maybe 'procmail'. Actually sending the final email is trivial, the core new thing that does the magic aggregation part is as simple or as complicated as needed, the NEEDS INFORMATION part is the commercial software configuration for its idea of sending email, the fiddly bit is dealing with that to get the emails commercial software sends into the new thing. The rest is a Simple Matter Of Programming (SMOP).
posted by zengargoyle at 7:28 AM on August 20, 2021
If you can run batch or at jobs and your email destination honors .forward files, it seems like a simple shell script could accept an email, and then either attach it to a previous email (which you could find through the batch queue); or, if there's no previous email, then this email starts a batch job that waits 5 minutes and forwards the email as desired.
If only one email comes in, it'll be delayed 5 minutes waiting for the next one. If a second email comes in while the first is waiting, then they'll be combined and go out immediately.
The only obscure part of this is batch processing -- people used to do it all the time, but I haven't seen it much lately. (I just tried on my MacOS...apparently, the daemon that runs batch jobs isn't enabled by default, but the infrastructure is there, so I'm sure I could turn it on.)
posted by spacewrench at 9:50 AM on August 20, 2021
If only one email comes in, it'll be delayed 5 minutes waiting for the next one. If a second email comes in while the first is waiting, then they'll be combined and go out immediately.
The only obscure part of this is batch processing -- people used to do it all the time, but I haven't seen it much lately. (I just tried on my MacOS...apparently, the daemon that runs batch jobs isn't enabled by default, but the infrastructure is there, so I'm sure I could turn it on.)
posted by spacewrench at 9:50 AM on August 20, 2021
Response by poster: Wow, guys! This is super cool. Sounding MUCH more doable than when I've previously tried Googling/asking online.
@TimHare unless I misunderstand, don't think this would work, as I need the emails to go FROM me (the software) TO other people ?
@zengargoyle - yes, you're right, outgoing mail. And again, yes, I do send regular emails but NOT via postfix. I send these directly from the gmail web interface. Postfix is sat between the commercial windows app and gmail because the CWA doesn't speak TLS so can't directly send to gmail without postfix as a relay (or MTA?)
The CWA sends email to a specific IP:port via SMTP - so yes postfix is listening locally on the LAN and the CWA has the IP of the postfix server in the "email server" settings field.
It's the SMOP you mention that I don't have the first clue how to do. As you can see from the fact I've setup a postfix server, I'm not 'not-technical' but I'm also not a programmer. I can bodge and glue together bits and bobs and use Google like the nearly best of them, but I can't from scratch program this solution. Sadly!
@spacewrench - this sounds....interesting...I'll do some googling re: batch processing and .forward files. It still sounds like there'd be a ways to go after that...but it's a totally unexplored avenue for me.
thanks!
posted by dance at 1:03 PM on August 20, 2021 [1 favorite]
@TimHare unless I misunderstand, don't think this would work, as I need the emails to go FROM me (the software) TO other people ?
@zengargoyle - yes, you're right, outgoing mail. And again, yes, I do send regular emails but NOT via postfix. I send these directly from the gmail web interface. Postfix is sat between the commercial windows app and gmail because the CWA doesn't speak TLS so can't directly send to gmail without postfix as a relay (or MTA?)
The CWA sends email to a specific IP:port via SMTP - so yes postfix is listening locally on the LAN and the CWA has the IP of the postfix server in the "email server" settings field.
It's the SMOP you mention that I don't have the first clue how to do. As you can see from the fact I've setup a postfix server, I'm not 'not-technical' but I'm also not a programmer. I can bodge and glue together bits and bobs and use Google like the nearly best of them, but I can't from scratch program this solution. Sadly!
@spacewrench - this sounds....interesting...I'll do some googling re: batch processing and .forward files. It still sounds like there'd be a ways to go after that...but it's a totally unexplored avenue for me.
thanks!
posted by dance at 1:03 PM on August 20, 2021 [1 favorite]
There's always programming. Unless somebody has done it before and had close enough to your situation to write something that you could use also.
So, there's a LAN with a Windows machine that runs the CWA, and an Ubuntu box somewhere listening on its LAN SMTP port and sends the mail out through Google.
I would do this:
Find a horribly simple SMTP server that does nothing but accept all mail and dump it in a mailbox. Configure it to run on a different port than postfix. Have the CWA send mail to that instead. Outgoing mail is now just sitting in a mailbox file on the *NIX box.
Write a script (using Perl natch... or whatever floats your boat) with the ever ubiquitous email/mailbox handling modules. It reads the mailbox, does the aggregation, and ends up with a list of new-messages that actually need to be sent. Each of those has a list of the original messages that contributed. Then it in turn sends each of the new-messages using the real postfix server, once sent it would then delete the contributory messages that made it up from the mailbox. Then it's done.
The rest is some systems admin stuff of mucking with systemd on the *NIX box to get the fake-SMTP-server started. And maybe the processing script if you want to go that way... I'd just throw it in a cron job.
Given the constraints of Crappy Windows Application (CWA) that does SMTP, the LAN, the *NIX box with postfix that does the TLS.... The answer is a magic smart proxy SMTP server that sits between the CWA and the postfix.
I think it would be non-trivial to get postfix to both accept mail and do hacky-aggregation-stuff *and* still be real SMTP to the world at the same time. This problem is not at either end where filters and forwards and procmail and stuff fixes things up. It's a middle problem that needs intervention in the middle: fake-SMTP/do stuff/real-SMTP. If for nothing else than separation of concerns/responsibility. Smart SMTP Proxy. Not sure if there is one already.
posted by zengargoyle at 1:12 AM on August 21, 2021
So, there's a LAN with a Windows machine that runs the CWA, and an Ubuntu box somewhere listening on its LAN SMTP port and sends the mail out through Google.
I would do this:
Find a horribly simple SMTP server that does nothing but accept all mail and dump it in a mailbox. Configure it to run on a different port than postfix. Have the CWA send mail to that instead. Outgoing mail is now just sitting in a mailbox file on the *NIX box.
Write a script (using Perl natch... or whatever floats your boat) with the ever ubiquitous email/mailbox handling modules. It reads the mailbox, does the aggregation, and ends up with a list of new-messages that actually need to be sent. Each of those has a list of the original messages that contributed. Then it in turn sends each of the new-messages using the real postfix server, once sent it would then delete the contributory messages that made it up from the mailbox. Then it's done.
The rest is some systems admin stuff of mucking with systemd on the *NIX box to get the fake-SMTP-server started. And maybe the processing script if you want to go that way... I'd just throw it in a cron job.
Given the constraints of Crappy Windows Application (CWA) that does SMTP, the LAN, the *NIX box with postfix that does the TLS.... The answer is a magic smart proxy SMTP server that sits between the CWA and the postfix.
I think it would be non-trivial to get postfix to both accept mail and do hacky-aggregation-stuff *and* still be real SMTP to the world at the same time. This problem is not at either end where filters and forwards and procmail and stuff fixes things up. It's a middle problem that needs intervention in the middle: fake-SMTP/do stuff/real-SMTP. If for nothing else than separation of concerns/responsibility. Smart SMTP Proxy. Not sure if there is one already.
posted by zengargoyle at 1:12 AM on August 21, 2021
Oh, debugging - How to setup a fake SMTP server to catch all mails? - Server Fault. Via something like dummy mock SMTP server - Google Search.
This combined with Python/Perl/whatever and a 'send email to SMTP server with X' search...
It shouldn't be hard to first create a totally transparent SMTP proxy that just sits in the middle receiving mail from the CWA and sending it to the MTA doing absolutely nothing. You get that working. Then you do things like holding and combining messages.
By the time you get this working and doing nothing, you're half-way there to doing your custom aggregation desire.
posted by zengargoyle at 4:08 AM on August 21, 2021
This combined with Python/Perl/whatever and a 'send email to SMTP server with X' search...
It shouldn't be hard to first create a totally transparent SMTP proxy that just sits in the middle receiving mail from the CWA and sending it to the MTA doing absolutely nothing. You get that working. Then you do things like holding and combining messages.
By the time you get this working and doing nothing, you're half-way there to doing your custom aggregation desire.
posted by zengargoyle at 4:08 AM on August 21, 2021
This thread is closed to new comments.
Milters are involved in discrete actions on messages one at a time.
Are there always two?
You might be able to have a milter stop a message, and hold it until a second comes along, and paste back in the first. But it would be essential that there are always two. This would take some python programming, or similar scripting.
posted by nickggully at 6:57 AM on August 20, 2021