Discussion:
Sending Bulk Mail - Own SMTP mail server on VPS vs 3rd Party Mail Service
Wen Lin via luv-main
2017-09-22 18:56:54 UTC
Permalink
One of the associations I'm in (besides LUV) have been experiencing
difficulties sending bulk emails to members on a monthly basis (for
announcements, newsletters purposes). This is because when you are trying
to send bulk emails via your free web mail account (in this case, GMail),
there is a limit imposed by Google (or any other webmail service providers)
on the number of email addresses we can mail out to, within a period of
time - and this limit can get ever stricter as time goes by. So the whole
mailout process becomes unmanageable after a while.

While the Ctte is looking into various alternatives, including several of
the 3rd party mail services / transactional mail services in the market, I
am offering them another option: Setup a SMTP mail server (e.g. Postfix)
on a VPS hosted by a web hosting company, and handle the whole bulk mailout
ourselves. I am willing to help them setting up a VPS, hardening the
Linux, installing & configuring all the mail software, etc, and then take
care of the regular maintenance.

For VPS hosting service - I had looked at one hostwinds.com (Editor's
Choice of au.pcmag.com 2017) (~ US$ 14 / mth). Will check out others to
compare - basically I'm after a reasonably priced and reputable VPS web
hosting company. I would like to hear any recommendation, or any
first-hand experience dealing with any particular web hosting company -
which I'm sure many of LUV members certainly have.

For domain name (.org.au) - About $48 for 2 years. Any reputable Domain
Registrar that you would recommend?

For SSL Certificate - I can install one from Let's Encrypt, which will be
free.

Any major cost/charges that I have missed?

I also intend to follow all the bulk mail best practices, SPF/DKIM
settings, MX records, security measures, email etiquette, etc, to make sure
that our organisation normal notification/newsletter to our members, would
not be treated as SPAM.

The 3rd party mail service providers that I had found so far: MailChimp,
MailJet, Amazon-SES, SendGrid, MailGun (and many others).

Did lots of google searches, trying to compare (Advantages & Disadvantages)
between managing own SMTP mail server and using 3rd party mail services.
The results were mix and varied.

I myself certainly have much more faith in opting for a self-managed mail
server solution over the commercial mail service ones.

Some of the advantages:

- Have full control over most aspect of the mailing process. More
flexible. Not constraint by the ever-changing rules imposed by 3rd party
mail services.
- Using all free & open source software - have all the freedom, and can
tap into the large FOSS Community out there for various kinds of help and
support.
- Cost-wise, the mail server option is generally on the lower end of the
price spectrum, as compared to many of the 3rd party mail services in the
market.
- No doubt the own mail server option would involve more work, more time
to manage the whole thing - but I'm quite happy to take up this challenge,
and to learn a lot from this hands-on experience.

I would be delighted if any of you who are experienced in managing a
linux-based mail server can kindly share with us some more real-life
examples and hands-on experiences - so that I can gather more solid
evidence & cases, hopefully to strengthen my case for a self-managed mail
server solution.

Some points that I had read somewhere that I would like to quote below to
seek your comment:

"*What a commercial mailing list operator does, and which you cannot easily
replicate, is build and maintain a reputation as a responsible and
reputable source of bulk email. If you are not going to be getting into
that, perhaps you could benefit from getting help from a commercial mailer.
... *

*Abuse handling, bounce handling etc are important to get right, but until
you have significant experience, your reputation is going to remain zero,
if not negative (which is a reasonable starting point for unknown domains
in this day and age). A lot of the reputation-based stuff like SPF, DKIM
etc will help only if you have a reputation to defend. On the other hand,
it does send the right signals to somebody who is deciding on whether or
not a sender is to be treated as reputable (or rather, their absence is not
a good sign; neither, in my book, is anything to suggest you are using
homegrown and/or prerelease software to send email). *..."

I also read somewhere which seemed to point to a 3rd option: Still use own
mail server, but direct the outbound mails to an external Mail Relay
service. What do you think?

When this whole project (if accepted by my Ctte) is done and implemented
successfully, I intend to share my experience in this whole process, in one
of LUV talks.

In the mean time, I have a lot of work to do, and lots to learn ...


Cheers,
Wen
Russell Coker via luv-main
2017-09-23 06:45:26 UTC
Permalink
Post by Wen Lin via luv-main
For VPS hosting service - I had looked at one hostwinds.com (Editor's
Choice of au.pcmag.com 2017) (~ US$ 14 / mth). Will check out others to
compare - basically I'm after a reasonably priced and reputable VPS web
hosting company. I would like to hear any recommendation, or any
first-hand experience dealing with any particular web hosting company -
which I'm sure many of LUV members certainly have.
https://www.linode.com/pricing

To just send out mail you don't need much from a hosting company. I've found
Linode to be very good and they have services starting at $5 per month that
will do what you want. 1G of RAM should be enough for sending mail.

https://www.scaleway.com/

Scaleway starts at E2.99 per month and has 2G of RAM.
Post by Wen Lin via luv-main
For domain name (.org.au) - About $48 for 2 years. Any reputable Domain
Registrar that you would recommend?
There are cheaper domains available, like .click.
Post by Wen Lin via luv-main
"*What a commercial mailing list operator does, and which you cannot easily
replicate, is build and maintain a reputation as a responsible and
reputable source of bulk email. If you are not going to be getting into
that, perhaps you could benefit from getting help from a commercial mailer.
... *
Don't send spam and don't use an IP address from a spam-friendly ISP and you
should be OK.
Post by Wen Lin via luv-main
I also read somewhere which seemed to point to a 3rd option: Still use own
mail server, but direct the outbound mails to an external Mail Relay
service. What do you think?
There's no point in that. Either run your own server or use an external
company.
--
My Main Blog http://etbe.coker.com.au/
My Documents Blog http://doc.coker.com.au/
Arjen Lentz via luv-main
2017-09-23 07:00:00 UTC
Permalink
Hi

For our VPS hosting needs in Australia we use Binary Lane. They're Australian owned as well, which I regard as an advantage (also in legal terms).
Stuff just works and their support has been very prompt and technically competent.

Linode is very good and we've used them for years, but they are American and thus directly subject to US legal leverage. Even if you used a Linode DC in Japan or Singapore, ultimately it would be open to US legal frivolities.
This may not be a consideration for some needs, but I'm just putting it out there.
Data sovereignty is a thing.

Regards,
Arjen.
Wen Lin via luv-main
2017-09-23 15:08:01 UTC
Permalink
Hi Arjen,

Thanks for your info.

I will check out this Binary Lane. BTW, I noticed a few of our LUV members
have VPS hosted by Digital Ocean. They seem quite active in hosting various
Linux related forums and tutorials. However, Digital Ocean is also
American, but they do have DCs outside of US, e.g. Canada.


Cheers,
Wen


On 23 Sep. 2017 5:00 pm, "Arjen Lentz" <***@lentz.com.au> wrote:

Hi

For our VPS hosting needs in Australia we use Binary Lane. They're
Australian owned as well, which I regard as an advantage (also in legal
terms).
Stuff just works and their support has been very prompt and technically
competent.

Linode is very good and we've used them for years, but they are American
and thus directly subject to US legal leverage. Even if you used a Linode
DC in Japan or Singapore, ultimately it would be open to US legal
frivolities.
This may not be a consideration for some needs, but I'm just putting it out
there.
Data sovereignty is a thing.

Regards,
Arjen.
Wen Lin via luv-main
2017-09-23 14:53:42 UTC
Permalink
Hi Russell,
Post by Wen Lin via luv-main
For VPS hosting service - I had looked at one hostwinds.com (Editor's
Choice of au.pcmag.com 2017) (~ US$ 14 / mth). Will check out others to
compare - basically I'm after a reasonably priced and reputable VPS web
hosting company. I would like to hear any recommendation, or any
first-hand experience dealing with any particular web hosting company -
which I'm sure many of LUV members certainly have.
https://www.linode.com/pricing

To just send out mail you don't need much from a hosting company. I've
found
Linode to be very good and they have services starting at $5 per month that
will do what you want. 1G of RAM should be enough for sending mail.

https://www.scaleway.com/

Scaleway starts at E2.99 per month and has 2G of RAM.
Post by Wen Lin via luv-main
For domain name (.org.au) - About $48 for 2 years. Any reputable Domain
Registrar that you would recommend?
I will check out the above 2 web hosting companies you had introduced.


There are cheaper domains available, like .click.


They already got a .org.au domain registered some time ago. So I just need
to find out where they had registered it, and then help renew it when it's
about to expire.

BTW, I thought I could use whois to find out the expiry date of this
domain, but I noticed that the registrars in Australia (for the .au) seem
to hide this date from the world. Apparently in Australia, the only way to
find out this info is to contact the registrar directly. Whereas for some
of the non-".au" domains, I could see the registration expiry date being
displayed. May be it's some sort of security measure in Australia.
Post by Wen Lin via luv-main
"*What a commercial mailing list operator does, and which you cannot easily
replicate, is build and maintain a reputation as a responsible and
reputable source of bulk email. If you are not going to be getting into
that, perhaps you could benefit from getting help from a commercial mailer.
... *
Don't send spam and don't use an IP address from a spam-friendly ISP and you
should be OK.


Very true.
Post by Wen Lin via luv-main
I also read somewhere which seemed to point to a 3rd option: Still use own
mail server, but direct the outbound mails to an external Mail Relay
service. What do you think?
There's no point in that. Either run your own server or use an external
company.


Thanks Russell for your input and advice.


Cheers,
Wen



--
My Main Blog http://etbe.coker.com.au/
My Documents Blog http://doc.coker.com.au/
Darren Wurf via luv-main
2017-09-23 23:04:54 UTC
Permalink
Post by Wen Lin via luv-main
I also intend to follow all the bulk mail best practices, SPF/DKIM
settings, MX records, security measures, email etiquette, etc, to make
sure that our organisation normal notification/newsletter to our
members, would not be treated as SPAM.
I use this docker container[1] to host a mail server on Binary Lane,
batteries are included. It took me some fiddling to get it working but I
took notes, happy to share them.

[1]: https://github.com/tomav/docker-mailserver
Jason White via luv-main
2017-09-24 20:35:19 UTC
Permalink
I've had good service from Linode as a hosting provider. SPF, DKIM and DMARC
took me quite a while to configure, attributable mostly to lack of experience.

I need to revisit my spam problem, yet again. The current solution is still
CRM114, which is keeping most of the spam at bay, but introducing too many
false positives.

I've tried rspamd in the past, as well as SpamAssassin, with worse results
than CRM114 is currently giving.

I've also successfully used Mailman on the server, but for a very low traffic
mailing list.
Peter Hall via luv-main
2017-09-24 05:03:59 UTC
Permalink
I set up a VPS to use mailman to send out a newsletter for a community
organisation.

Ransom IT http://ransomit.com.au/ is my host. They are one of the cheaper
options if you want your server to be located in Australia. I haven't
noticed any problems, though sending a monthly newsletter doesn't take much
resources. When I signed up their ToS had a clause about no bulk mail,
however I contacted them and they said it was a no spam policy and they are
OK with opt-in newsletters. It looks like they've since revised the ToS to
make that clear.

I use Gandi https://www.gandi.net/en for DNS. Reputable, low cost, supports
two factor authentication, has an API if you ever want to automate anything.

Let's Encrypt worked for me for HTTPS certificates.

Mail Tester http://www.mail-tester.com/ was very useful when I was first
checking to make sure I had all the anti-spam features set up correctly.

It was my first time setting up a mail server. I found the Ars Technica
guide extremely useful:
https://arstechnica.com/information-technology/2014/02/how-to-run-your-own-e-mail-server-with-your-own-domain-part-1/
https://arstechnica.com/information-technology/2014/03/taking-e-mail-back-part-2-arming-your-server-with-postfix-dovecot/
https://arstechnica.com/information-technology/2014/03/taking-e-mail-back-part-3-fortifying-your-box-against-spammers/
https://arstechnica.com/information-technology/2014/04/taking-e-mail-back-part-4-the-finale-with-webmail-everything-after/

The guide is a few years old now, but AFAIK it is still worth reading.
Email hasn't fundamentally changed in the last 3 years.

I use mailman https://www.gnu.org/software/mailman/index.html to manage
mailing lists. It took a while to get it configured nicely, the default
settings tend to mangle mail in a way that isn't compatible with modern
anti-spam features.

Cheers,
Peter
Post by Wen Lin via luv-main
One of the associations I'm in (besides LUV) have been experiencing
difficulties sending bulk emails to members on a monthly basis (for
announcements, newsletters purposes). This is because when you are trying
to send bulk emails via your free web mail account (in this case, GMail),
there is a limit imposed by Google (or any other webmail service providers)
on the number of email addresses we can mail out to, within a period of
time - and this limit can get ever stricter as time goes by. So the whole
mailout process becomes unmanageable after a while.
While the Ctte is looking into various alternatives, including several of
the 3rd party mail services / transactional mail services in the market, I
am offering them another option: Setup a SMTP mail server (e.g. Postfix)
on a VPS hosted by a web hosting company, and handle the whole bulk mailout
ourselves. I am willing to help them setting up a VPS, hardening the
Linux, installing & configuring all the mail software, etc, and then take
care of the regular maintenance.
For VPS hosting service - I had looked at one hostwinds.com (Editor's
Choice of au.pcmag.com 2017) (~ US$ 14 / mth). Will check out others to
compare - basically I'm after a reasonably priced and reputable VPS web
hosting company. I would like to hear any recommendation, or any
first-hand experience dealing with any particular web hosting company -
which I'm sure many of LUV members certainly have.
For domain name (.org.au) - About $48 for 2 years. Any reputable Domain
Registrar that you would recommend?
For SSL Certificate - I can install one from Let's Encrypt, which will be
free.
Any major cost/charges that I have missed?
I also intend to follow all the bulk mail best practices, SPF/DKIM
settings, MX records, security measures, email etiquette, etc, to make sure
that our organisation normal notification/newsletter to our members, would
not be treated as SPAM.
The 3rd party mail service providers that I had found so far: MailChimp,
MailJet, Amazon-SES, SendGrid, MailGun (and many others).
Did lots of google searches, trying to compare (Advantages &
Disadvantages) between managing own SMTP mail server and using 3rd party
mail services. The results were mix and varied.
I myself certainly have much more faith in opting for a self-managed mail
server solution over the commercial mail service ones.
- Have full control over most aspect of the mailing process. More
flexible. Not constraint by the ever-changing rules imposed by 3rd party
mail services.
- Using all free & open source software - have all the freedom, and
can tap into the large FOSS Community out there for various kinds of help
and support.
- Cost-wise, the mail server option is generally on the lower end of
the price spectrum, as compared to many of the 3rd party mail services in
the market.
- No doubt the own mail server option would involve more work, more
time to manage the whole thing - but I'm quite happy to take up this
challenge, and to learn a lot from this hands-on experience.
I would be delighted if any of you who are experienced in managing a
linux-based mail server can kindly share with us some more real-life
examples and hands-on experiences - so that I can gather more solid
evidence & cases, hopefully to strengthen my case for a self-managed mail
server solution.
Some points that I had read somewhere that I would like to quote below to
"*What a commercial mailing list operator does, and which you cannot
easily replicate, is build and maintain a reputation as a responsible and
reputable source of bulk email. If you are not going to be getting into
that, perhaps you could benefit from getting help from a commercial mailer.
... *
*Abuse handling, bounce handling etc are important to get right, but until
you have significant experience, your reputation is going to remain zero,
if not negative (which is a reasonable starting point for unknown domains
in this day and age). A lot of the reputation-based stuff like SPF, DKIM
etc will help only if you have a reputation to defend. On the other hand,
it does send the right signals to somebody who is deciding on whether or
not a sender is to be treated as reputable (or rather, their absence is not
a good sign; neither, in my book, is anything to suggest you are using
homegrown and/or prerelease software to send email). *..."
I also read somewhere which seemed to point to a 3rd option: Still use
own mail server, but direct the outbound mails to an external Mail Relay
service. What do you think?
When this whole project (if accepted by my Ctte) is done and implemented
successfully, I intend to share my experience in this whole process, in one
of LUV talks.
In the mean time, I have a lot of work to do, and lots to learn ...
Cheers,
Wen
_______________________________________________
luv-main mailing list
https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main
--
Trapped in signature factory please send help
Arjen Lentz via luv-main
2017-09-24 21:49:41 UTC
Permalink
Hi Peter
Post by Peter Hall via luv-main
I use mailman https://www.gnu.org/software/mailman/index.html to manage
mailing lists. It took a while to get it configured nicely, the default
settings tend to mangle mail in a way that isn't compatible with modern
anti-spam features.
Do you have any docu on that?
That'd be great.

That's per list also, isn't it, so any new list would have to be set up in the right way again.

Regards,
Arjen.
Jason White via luv-main
2017-09-24 22:08:26 UTC
Permalink
Post by Arjen Lentz via luv-main
Do you have any docu on that?
That'd be great.
Here's the official page:
https://wiki.list.org/DEV/DMARC

In my case, I turned off features of Mailman that modified the subject lines
or bodies of outbound messages, thus preventing it from breaking DKIM
signatures.
Russell Coker via luv-main
2017-09-25 00:54:03 UTC
Permalink
Post by Jason White via luv-main
Post by Arjen Lentz via luv-main
Do you have any docu on that?
That'd be great.
https://wiki.list.org/DEV/DMARC
In my case, I turned off features of Mailman that modified the subject lines
or bodies of outbound messages, thus preventing it from breaking DKIM
signatures.
https://doc.coker.com.au/internet/dkim-and-mailing-lists/

That doesn't work. Unless you can get everyone who sends to the list to use
the "relaxed" option you will have Mailman header munging breaking thigns.
Getting every sender to do things the way you desire isn't a viable option.

Mailman also sometimes rewrites the body for a different encoding.

I tried to get Mailman to pass signed messages through on this list, it wasn't
possible.
--
My Main Blog http://etbe.coker.com.au/
My Documents Blog http://doc.coker.com.au/
Peter Hall via luv-main
2017-10-01 08:19:06 UTC
Permalink
No doco, sorry. The list I run is a single sender announce list, so getting
all senders to use the right options was practical for me.

IIRC the main change was to get mailman to rewrite headers so that messages
came from the list address. Trying to keep the original sender caused some
problem that I never got to the bottom of.
--
Trapped in signature factory please send help
Post by Russell Coker via luv-main
Post by Jason White via luv-main
Post by Arjen Lentz via luv-main
Do you have any docu on that?
That'd be great.
https://wiki.list.org/DEV/DMARC
In my case, I turned off features of Mailman that modified the subject
lines
Post by Jason White via luv-main
or bodies of outbound messages, thus preventing it from breaking DKIM
signatures.
https://doc.coker.com.au/internet/dkim-and-mailing-lists/
That doesn't work. Unless you can get everyone who sends to the list to use
the "relaxed" option you will have Mailman header munging breaking thigns.
Getting every sender to do things the way you desire isn't a viable option.
Mailman also sometimes rewrites the body for a different encoding.
I tried to get Mailman to pass signed messages through on this list, it wasn't
possible.
--
My Main Blog http://etbe.coker.com.au/
My Documents Blog http://doc.coker.com.au/
_______________________________________________
luv-main mailing list
https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main
Loading...