On 2/1/2011 7:43 AM, sunhux G wrote:
>
> Our current way of blocking a spam address is by editing
> access_sender & access_recipient & then reload postmap.
>
> From time to time we're given addresses that should never
> be blocked but due to staff turnover & documentation not
> up-to-date, an address that should never be blocked was
> somehow blocked.
>
> Pardon me if this has been discussed before,
> what's the best way to go about preventing such mistakes?
>
> Is there a whitelist file that we can enter addresses that should
> never blocked so that even if this address is manually added
> into access_sender & access_recipient, they will still not be
> blocked (& possibly will be automatically removed from the
> two files access_sender/recipient).
>
> If there's such a whitelist file, presumably there should be 2
> of them, one for sending & receiving.  Let me know the full
> directory path & filename of the whitelist files
>
Postfix does not allocate certain file names for access maps.
You may have as many as you like, the only thing that matters is the
order of the maps in the restriction class.
The first match always wins, so put your whitelists before any blacklists.

I recommend using "permit_auth_destination" as the result for a
whitelist due to your mentioned turnover rate.
This will prevent any open relays if the whitelist is incorrectly placed
in the chain of restrictions (in recipient restrictions before
reject_unauth_destination)

Zitat von Wietse Venema <wietse@...>:

> lst_hoe02@...:
>> Hello
>>
>> we are on the way to IPv6 and some question arise about Postfix IPv6
>> behaviour in dual stack setup.
>
> You mean dual-protocol. Unlike some versions of Linux, there exist
> systems that have a single unified TCP/IP stack implementation (the
> protocols have a fair amount of behavior in common).
>
>> 1.) Do the lookups for AAAA when resolving MX records occur in
>> parallel to A queries, or is some additional latency expected due to
>> the fact that many AAAA queries fail by timeout and others resolving
>> errors?
>
> With smtp_host_lookup=dns, all SMTP client lookups are sequential.
> With smtp_host_lookup=native (or dns,native) Postfix in dual-protocol
> mode will invoke getaddrinfo() with hints.ai_family=PF_UNSPEC.
> The internals of getaddrinfo() are system dependent.
>
>> 2.) Do AAAA/ip6.arpa lookup occur on client connects from IPv4
>> addresses or only when a client connects by IPv6 (name/reverse lookups)?
>
> The Postfix SMTP server's FCRDNS lookups use the getnameinfo() and
> getaddrinfo() system library functions. In dual-protocol mode,
> Postfix invokes getaddrinfo() with hints.ai_family=PF_UNSPEC; it
> would be smarter to pass the client's address family instead.
>
>  Wietse

Is the last point still on todo list or considered worth to be
included in the future?

Regards

Andreas

Hi there. Hi, I've been googling around all morning and I'm completely ignorant on what I'm going to ask, so please forgive me if I make no sense. I have 2 independent servers running postifx+mysql+(other_things) all controlled from a nice web interfacce called ISPConfig3. Those 2 servers are completely independent with many domains configured in each of them. Authentication is done against each server's separate and different mysql database. I'm testing Perdition for imap and pop3 connections so webmail access is more consistent/unified, and in case of customers with email services in both servers, we make it easier for them since the proxy redirects connections to the right imap server. My question: is there such a similar product (SMTP proxy) that can be configured in the same way to hide the real smtp servers and deliver/accept_mail_from_our_2_different_pools_of_users using the correct server?

TIA

Ignacio

On Tue, Feb 01, 2011 at 05:33:14PM +0100, Ignacio Garcia wrote:

> Hi there. Hi, I've been googling around all morning and I'm
> completely ignorant on what I'm going to ask, so please forgive me if I
> make no sense. I have 2 independent servers running
> postifx+mysql+(other_things) all controlled from a nice web interfacce
> called ISPConfig3. Those 2 servers are completely independent with many
> domains configured in each of them. Authentication is done against each
> server's separate and different mysql database. I'm testing Perdition
> for imap and pop3 connections so webmail access is more
> consistent/unified, and in case of customers with email services in both
> servers, we make it easier for them since the proxy redirects
> connections to the right imap server. My question: is there such a
> similar product (SMTP proxy) that can be configured in the same way to
> hide the real smtp servers and
> deliver/accept_mail_from_our_2_different_pools_of_users using the
> correct server?

Well, the proxy won't know what to do before the user authenticates,
and you say the the authentication databases are split, so it is far
from clear how you expect this could work.

However, if Perdition presents a unified IMAP interface, you could
perhaps use an "rimap" backend with Cyrus SASL to authenticate the
user.

I am not aware of any SMTP proxies whose downstream SMTP server is
selected after user authentication. It is probably easiest to just
operate a unified submission server that authenticates the union of the
two sets of users, and then routes to the right server via sender-based
routing. In other-words, not a proxy but a store-and-forward MSA.

Postfix can do that.

--
	 Viktor.

On 02/01/2011 05:43 PM, Victor Duchovni wrote:
> On Tue, Feb 01, 2011 at 05:33:14PM +0100, Ignacio Garcia wrote:
>
>> Hi there. Hi, I've been googling around all morning and I'm
>> completely ignorant on what I'm going to ask, so please forgive me if I
>> make no sense. I have 2 independent servers running
>> postifx+mysql+(other_things) all controlled from a nice web interfacce
>> called ISPConfig3. Those 2 servers are completely independent with many
>> domains configured in each of them. Authentication is done against each
>> server's separate and different mysql database. I'm testing Perdition
>> for imap and pop3 connections so webmail access is more
>> consistent/unified, and in case of customers with email services in both
>> servers, we make it easier for them since the proxy redirects
>> connections to the right imap server. My question: is there such a
>> similar product (SMTP proxy) that can be configured in the same way to
>> hide the real smtp servers and
>> deliver/accept_mail_from_our_2_different_pools_of_users using the
>> correct server?
> Well, the proxy won't know what to do before the user authenticates,
> and you say the the authentication databases are split, so it is far
> from clear how you expect this could work.
>
> However, if Perdition presents a unified IMAP interface, you could
> perhaps use an "rimap" backend with Cyrus SASL to authenticate the
> user.
>
> I am not aware of any SMTP proxies whose downstream SMTP server is
> selected after user authentication. It is probably easiest to just
> operate a unified submission server that authenticates the union of the
> two sets of users, and then routes to the right server via sender-based
> routing. In other-words, not a proxy but a store-and-forward MSA.
>
> Postfix can do that.
>
There is a mysql proxy software , maybe it could help to authenticate
users from both databases ?

http://forge.mysql.com/wiki/MySQL_Proxy

Am 01.02.2011 17:51, schrieb Frank Bonnet:
> On 02/01/2011 05:43 PM, Victor Duchovni wrote:
>> On Tue, Feb 01, 2011 at 05:33:14PM +0100, Ignacio Garcia wrote:
>>
>>> Hi there. Hi, I've been googling around all morning and I'm
>>> completely ignorant on what I'm going to ask, so please forgive me if I
>>> make no sense. I have 2 independent servers running
>>> postifx+mysql+(other_things) all controlled from a nice web interfacce
>>> called ISPConfig3. Those 2 servers are completely independent with many
>>> domains configured in each of them. Authentication is done against each
>>> server's separate and different mysql database. I'm testing Perdition
>>> for imap and pop3 connections so webmail access is more
>>> consistent/unified, and in case of customers with email services in both
>>> servers, we make it easier for them since the proxy redirects
>>> connections to the right imap server. My question: is there such a
>>> similar product (SMTP proxy) that can be configured in the same way to
>>> hide the real smtp servers and
>>> deliver/accept_mail_from_our_2_different_pools_of_users using the
>>> correct server?
>> Well, the proxy won't know what to do before the user authenticates,
>> and you say the the authentication databases are split, so it is far
>> from clear how you expect this could work.
>>
>> However, if Perdition presents a unified IMAP interface, you could
>> perhaps use an "rimap" backend with Cyrus SASL to authenticate the
>> user.
>>
>> I am not aware of any SMTP proxies whose downstream SMTP server is
>> selected after user authentication. It is probably easiest to just
>> operate a unified submission server that authenticates the union of the
>> two sets of users, and then routes to the right server via sender-based
>> routing. In other-words, not a proxy but a store-and-forward MSA.
>>
>> Postfix can do that.
>>
> There is a mysql proxy software , maybe it could help to authenticate
> users from both databases ?
>
> http://forge.mysql.com/wiki/MySQL_Proxy

What about using only one mysql-server and the other one as
replication slave, postfix is readonly

So you would have the backend which writes to the master
and the load from the smtp-servers is spread and as
benefit if the master dies there is a additional backup

lst_hoe02@...:
> Zitat von Wietse Venema <wietse@...>:
>
> > lst_hoe02@...:
> >> Hello
> >>
> >> we are on the way to IPv6 and some question arise about Postfix IPv6
> >> behaviour in dual stack setup.
> >
> > You mean dual-protocol. Unlike some versions of Linux, there exist
> > systems that have a single unified TCP/IP stack implementation (the
> > protocols have a fair amount of behavior in common).
> >
> >> 1.) Do the lookups for AAAA when resolving MX records occur in
> >> parallel to A queries, or is some additional latency expected due to
> >> the fact that many AAAA queries fail by timeout and others resolving
> >> errors?
> >
> > With smtp_host_lookup=dns, all SMTP client lookups are sequential.
> > With smtp_host_lookup=native (or dns,native) Postfix in dual-protocol
> > mode will invoke getaddrinfo() with hints.ai_family=PF_UNSPEC.
> > The internals of getaddrinfo() are system dependent.
> >
> >> 2.) Do AAAA/ip6.arpa lookup occur on client connects from IPv4
> >> addresses or only when a client connects by IPv6 (name/reverse lookups)?
> >
> > The Postfix SMTP server's FCRDNS lookups use the getnameinfo() and
> > getaddrinfo() system library functions. In dual-protocol mode,
> > Postfix invokes getaddrinfo() with hints.ai_family=PF_UNSPEC; it
> > would be smarter to pass the client's address family instead.
> >
> >  Wietse
>
> Is the last point still on todo list or considered worth to be
> included in the future?

It is not a high priority. Moreover, this means ripping up a
low-level API, so it would take longer. I'm still fixing code
that was merged into Postfix 9 years ago.

	 Wietse

On Tue, 1 Feb 2011 11:43:25 -0500, Victor Duchovni
<Victor.Duchovni@...> wrote:
> On Tue, Feb 01, 2011 at 05:33:14PM +0100, Ignacio Garcia wrote:
>
>> Hi there. Hi, I've been googling around all morning and I'm
>> completely ignorant on what I'm going to ask, so please forgive me if I
>> make no sense. I have 2 independent servers running
>> postifx+mysql+(other_things) all controlled from a nice web interfacce
>> called ISPConfig3. Those 2 servers are completely independent with many
>> domains configured in each of them. Authentication is done against each
>> server's separate and different mysql database. I'm testing Perdition
>> for imap and pop3 connections so webmail access is more
>> consistent/unified, and in case of customers with email services in both
>> servers, we make it easier for them since the proxy redirects
>> connections to the right imap server. My question: is there such a
>> similar product (SMTP proxy) that can be configured in the same way to
>> hide the real smtp servers and
>> deliver/accept_mail_from_our_2_different_pools_of_users using the
>> correct server?
>
> Well, the proxy won't know what to do before the user authenticates,
> and you say the the authentication databases are split, so it is far
> from clear how you expect this could work.
>
> However, if Perdition presents a unified IMAP interface, you could
> perhaps use an "rimap" backend with Cyrus SASL to authenticate the
> user.
>
> I am not aware of any SMTP proxies whose downstream SMTP server is
> selected after user authentication. It is probably easiest to just
> operate a unified submission server that authenticates the union of the
> two sets of users, and then routes to the right server via sender-based
> routing. In other-words, not a proxy but a store-and-forward MSA.
>
> Postfix can do that.

Victor, thanks for your quick response.

yes, I did not take into account that authentication does not always
take place in SMTP. So I guess that leaves me with no other option but
to consider a round robin setup. However, I'm not sure how this works.
Do I need to setup each postfix server to accept messages from/to both
sets of users, or in this scenario, if the first connection fails, it'll
try the second automagically?

Can anybody point me to a tutorail, howto, etc. on how to setup postfix
in a round robin environment?

Thanks so much.

Ignacio

On Tue, Feb 01, 2011 at 06:11:58PM +0100, Ignacio Garcia wrote:

> > However, if Perdition presents a unified IMAP interface, you could
> > perhaps use an "rimap" backend with Cyrus SASL to authenticate the
> > user.
> >
> > I am not aware of any SMTP proxies whose downstream SMTP server is
> > selected after user authentication. It is probably easiest to just
> > operate a unified submission server that authenticates the union of the
> > two sets of users, and then routes to the right server via sender-based
> > routing. In other-words, not a proxy but a store-and-forward MSA.
> >
> > Postfix can do that.
>
> Victor, thanks for your quick response.
>
> yes, I did not take into account that authentication does not always
> take place in SMTP.

It does if you require it, as is normal with a submission service.

> So I guess that leaves me with no other option but
> to consider a round robin setup.

No, you can create a unified submission server which accepts submissions
for both environments, and forwards to the right server for outbound
delivery.

> However, I'm not sure how this works.
> Do I need to setup each postfix server to accept messages from/to both
> sets of users, or in this scenario, if the first connection fails, it'll
> try the second automagically?

> Can anybody point me to a tutorail, howto, etc. on how to setup postfix
> in a round robin environment?

You are not thinking very clearly yet. You must distinguish clearly
between:

     - Submission, users submitting mail for outgoing delivery. This is
       visible to users, since they set the server in question as their
       MUAs "SMTP server". This needs its own design, and it is what I
       thought you wanted help with.

     - Inbound MX service. Just publish appropriate MX records, and add
       front-end SMTP servers to each of the two environments as necessary.
       In ISP-grade implementations the SMTP inbound servers are not also
       IMAP servers, rather they forward mail to IMAP servers (via LMTP
       or SMTP).

Which do you need help with? State clear requirements for either or
each problem, and work from there.

--
	 Viktor.

> You are not thinking very clearly yet. You must distinguish clearly
> between:
>
>     - Submission, users submitting mail for outgoing delivery. This is
>       visible to users, since they set the server in question as their
>       MUAs "SMTP server". This needs its own design, and it is what I
>       thought you wanted help with.
>
>     - Inbound MX service. Just publish appropriate MX records, and add
>       front-end SMTP servers to each of the two environments as necessary.
>       In ISP-grade implementations the SMTP inbound servers are not also
>       IMAP servers, rather they forward mail to IMAP servers (via LMTP
>       or SMTP).
>
> Which do you need help with? State clear requirements for either or
> each problem, and work from there.


yes, you're right, sorry. Maybe if I tell you what I want to do I can
make myself clearer. We have both a submission service and inbound mx
service. we want to have a unified smtp.mycompany.com so all
"submissions" can be processed using this canonical domain name. i
believe that is not too dificult. We also want to run both servers so 1
is mx-backup of the other and viceversa. However, I forsee several
problems here:

- both servers need to check against both databases for valid
destinations
- each server must know if delivery is to be local or it has to be
relayed to the other server.
- one server virtual_transport=maildrop (to courier-imap), the other
=dovecot (we are going with dovecot for the future, the one with courier
is older, but plenty of users)

So, is there any hope?

Ignacio

On Tue, Feb 01, 2011 at 06:50:04PM +0100, Ignacio Garcia wrote:

> > Which do you need help with? State clear requirements for either or
> > each problem, and work from there.
>
>
> yes, you're right, sorry. Maybe if I tell you what I want to do I can
> make myself clearer. We have both a submission service and inbound mx
> service. we want to have a unified smtp.mycompany.com so all
> "submissions" can be processed using this canonical domain name. i
> believe that is not too dificult.

Good if you know how to handle submission, you're half way there...

> We also want to run both servers so 1
> is mx-backup of the other and viceversa.

This splits into two use-cases:

     * Relay

    	 - Each server accepts mail for the other's domains, and relays
	   them to the right server. All you need is access to a table
	   of valid recipients for the relay domains, and table of
	   said domain names.

     * Full-service:

    	 - Each server accepts mail for all the domains, and delivers
	   to the correct mail store. Need fully unified data model.

Which use-case are you aiming for?

> - both servers need to check against both databases for valid
> destinations

Correct (s/destinations/user-addresses/).

> - each server must know if delivery is to be local or it has to be
> relayed to the other server.

Postfix does that automatically. Just add the domain to "relay_domains"
and not to virtual_mailbox_domains, or similar.

> - one server virtual_transport=maildrop (to courier-imap), the other
> =dovecot (we are going with dovecot for the future, the one with courier
> is older, but plenty of users)

The difference in final delivery mechanisms is immaterial.

--
	 Viktor.

> This splits into two use-cases:
>
>     * Relay
>
>      - Each server accepts mail for the other's domains, and relays
> 	  them to the right server. All you need is access to a table
> 	  of valid recipients for the relay domains, and table of
> 	  said domain names.
>
>     * Full-service:
>
>      - Each server accepts mail for all the domains, and delivers
> 	  to the correct mail store. Need fully unified data model.
>
> Which use-case are you aiming for?
>

definitely full-service. I do not understand whan you say that I need a
fully unified data model.


>> - both servers need to check against both databases for valid
>> destinations
>
> Correct (s/destinations/user-addresses/).

ypu're right.

>> - each server must know if delivery is to be local or it has to be
>> relayed to the other server.
>
> Postfix does that automatically. Just add the domain to "relay_domains"
> and not to virtual_mailbox_domains, or similar.

ok.

>> - one server virtual_transport=maildrop (to courier-imap), the other
>> =dovecot (we are going with dovecot for the future, the one with courier
>> is older, but plenty of users)
>
> The difference in final delivery mechanisms is immaterial.

Great!!!


So, can you point me to the right direction in learning how to get the
full-service configuration done?

Thanks!!

Ignacio

On Tue, Feb 01, 2011 at 07:11:52PM +0100, Ignacio Garcia wrote:

> > This splits into two use-cases:
> >
> >     * Relay
> >
> >      - Each server accepts mail for the other's domains, and relays
> > 	  them to the right server. All you need is access to a table
> > 	  of valid recipients for the relay domains, and table of
> > 	  said domain names.
> >
> >     * Full-service:
> >
> >      - Each server accepts mail for all the domains, and delivers
> > 	  to the correct mail store. Need fully unified data model.
> >
> > Which use-case are you aiming for?
> >
>
> definitely full-service. I do not understand whan you say that I need a
> fully unified data model.

In addition to recipient lists, you need mailstore locations, and
transport settings, ... to actually deliver the mail.

> >> - both servers need to check against both databases for valid
> >> destinations
> >
> > Correct (s/destinations/user-addresses/).
>
> ypu're right.
>
> >> - each server must know if delivery is to be local or it has to be
> >> relayed to the other server.
> >
> > Postfix does that automatically. Just add the domain to "relay_domains"
> > and not to virtual_mailbox_domains, or similar.
>
> ok.

This is for the "relay" use-case.

> >> - one server virtual_transport=maildrop (to courier-imap), the other
> >> =dovecot (we are going with dovecot for the future, the one with courier
> >> is older, but plenty of users)
> >
> > The difference in final delivery mechanisms is immaterial.
>
> Great!!!

Sorry, for the full-service use case, both sets of SMTP servers
need to support both delivery mechanisms and choose the correct
one for each user.

Ideally it would be LMTP for both, and the IMAP servers would handle
the delivery logistics.

> So, can you point me to the right direction in learning how to get the
> full-service configuration done?

Learn about virtual_alias rewriting or per-user transports in
ADDRESS_REWRITING_README.html. Strongly consider using LMTP for
all MTA -> Mail store deliveries, so that the MTA does not need
to understand the internal architecture of the IMAP store.

--
	 Viktor.

Viktor, thanks very much. I'll be reading about all this and hopefully
will be asking some more questions once I have more knowledge of this.

Ignacio


On Tue, 1 Feb 2011 13:20:07 -0500, Victor Duchovni
<Victor.Duchovni@...> wrote:
> On Tue, Feb 01, 2011 at 07:11:52PM +0100, Ignacio Garcia wrote:
>
>> > This splits into two use-cases:
>> >
>> >     * Relay
>> >
>> >      - Each server accepts mail for the other's domains, and relays
>> > 	  them to the right server. All you need is access to a table
>> > 	  of valid recipients for the relay domains, and table of
>> > 	  said domain names.
>> >
>> >     * Full-service:
>> >
>> >      - Each server accepts mail for all the domains, and delivers
>> > 	  to the correct mail store. Need fully unified data model.
>> >
>> > Which use-case are you aiming for?
>> >
>>
>> definitely full-service. I do not understand whan you say that I need a
>> fully unified data model.
>
> In addition to recipient lists, you need mailstore locations, and
> transport settings, ... to actually deliver the mail.
>
>> >> - both servers need to check against both databases for valid
>> >> destinations
>> >
>> > Correct (s/destinations/user-addresses/).
>>
>> ypu're right.
>>
>> >> - each server must know if delivery is to be local or it has to be
>> >> relayed to the other server.
>> >
>> > Postfix does that automatically. Just add the domain to "relay_domains"
>> > and not to virtual_mailbox_domains, or similar.
>>
>> ok.
>
> This is for the "relay" use-case.
>
>> >> - one server virtual_transport=maildrop (to courier-imap), the other
>> >> =dovecot (we are going with dovecot for the future, the one with courier
>> >> is older, but plenty of users)
>> >
>> > The difference in final delivery mechanisms is immaterial.
>>
>> Great!!!
>
> Sorry, for the full-service use case, both sets of SMTP servers
> need to support both delivery mechanisms and choose the correct
> one for each user.
>
> Ideally it would be LMTP for both, and the IMAP servers would handle
> the delivery logistics.
>
>> So, can you point me to the right direction in learning how to get the
>> full-service configuration done?
>
> Learn about virtual_alias rewriting or per-user transports in
> ADDRESS_REWRITING_README.html. Strongly consider using LMTP for
> all MTA -> Mail store deliveries, so that the MTA does not need
> to understand the internal architecture of the IMAP store.

--
Ignacio Garcia
Gerente
E.S. Oenus SL
Tel: 962 961 017
SkypeID: oenus.com
http://www.oenus.com

On 2/1/2011 12:50 PM, Ignacio Garcia wrote:
>> You are not thinking very clearly yet. You must distinguish clearly
>> between:
>>
>>      - Submission, users submitting mail for outgoing delivery. This is
>>        visible to users, since they set the server in question as their
>>        MUAs "SMTP server". This needs its own design, and it is what I
>>        thought you wanted help with.
>>
>>      - Inbound MX service. Just publish appropriate MX records, and add
>>        front-end SMTP servers to each of the two environments as necessary.
>>        In ISP-grade implementations the SMTP inbound servers are not also
>>        IMAP servers, rather they forward mail to IMAP servers (via LMTP
>>        or SMTP).
>>
>> Which do you need help with? State clear requirements for either or
>> each problem, and work from there.
>
> yes, you're right, sorry. Maybe if I tell you what I want to do I can
> make myself clearer. We have both a submission service and inbound mx
> service. we want to have a unified smtp.mycompany.com so all
> "submissions" can be processed using this canonical domain name. i
> believe that is not too dificult. We also want to run both servers so 1
> is mx-backup of the other and viceversa. However, I forsee several
> problems here:
>
> - both servers need to check against both databases for valid
> destinations
> - each server must know if delivery is to be local or it has to be
> relayed to the other server.
> - one server virtual_transport=maildrop (to courier-imap), the other
> =dovecot (we are going with dovecot for the future, the one with courier
> is older, but plenty of users)
>
> So, is there any hope?
>
> Ignacio
Speaking not as a Postfix expert but as a DBA, and speaking more on
administrative strategy rather than technically, I would consider
merging the two databases. This is just a guess that the two servers are
running under the same organizational umbrella and that the DB's serve a
similar purpose for a single organization. Ugliness will grow and grow
around accommodating two databases that should be one.

The unified DB would have a "server_a_user_table" and a
"server_b_user_table" ([much] better yet, a unified table with a server
tag column so arbitrary servers can be added ultimately). Then you have
the flexibility to distinguish a local user lookup versus a relay lookup
on each server by configuring each of transport queries appropriately.
(Hint:
http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html#function_if)

Do the painful merging steps now and grow a sensible architecture that
can last for years. All kinds of dust will get rooted out in the
process. Optimistically your business will only need to add servers. An
obvious initial benefit is the simplified backup and monitoring
requirements.

Then as others have mentioned, the MySQL architecture can be replicated,
tuned, and accessed remotely with appropriate security settings. It can
even be replicated while live by setting it to read-only during initial
replication.

I believe efficiency in a replicated setting will still be very high.
Mail is very heavily read-oriented for DB's: settings/user migration/new
users don't change that much but there are 10^n lookups a day.

-Daniel

On Tue, Feb 01, 2011 at 08:43:41PM +0800, sunhux G wrote:
> Subject: foolproof whitelisting

Hmmm, whitelisting is often foolish, or foolishly implemented, so
this is difficult. :)

> Our current way of blocking a spam address is by editing
> access_sender & access_recipient & then reload postmap.

As Brian pointed out, those filenames have no universal meaning in
Postfixology. I would assume that you're using access_sender as a
check_sender_access lookup, and access_recipient to lookup
check_recipient_access, but I have seen cases posted on this list
where that was not so, and the name had nothing to do with the use of
the file. Or sometimes that the file was not even being used! (That's
why the list guidelines ask for postconf -n and relevant logs with
questions.)

Now, I'd go a step further and look at your wording, "blocking a spam
address." What does this mean? A sender address which "sent spam"?

Did you know that sender addresses are nearly meaningless in spam?
The vast majority of spam is sent with forged addresses. Sometimes
these are real addresses. In general, it's the wrong approach to be
blocking "spam senders" by sender address, because:
     1. That address might be legitimately used by a non-spammer; and
     2. Blocking that address is usually not going to be effective,
        because subsequent spam runs will use different sender
        addresses.

Yes, many thousands of freemail accounts have been set up by and for
the use of spammers, and in that case (when the spam originates from
that freemail provider) such check_sender_access blocking can be
effective. But this accounts for a small portion of the spam I see.

The best introductory tutorial to spam abatement I know is the one
offered by Spamhaus:
     http://www.spamhaus.org/whitepapers/effective_filtering.html

In Postfix terms, you would want to do reject_rbl_client using
zen.spamhaus.org, and reject_rhsbl_{client,sender,helo} using
dbl.spamhaus.org.

Also, not mentioned by the Spamhaus document, simple HELO checks are
very effective. I find that reject_non_fqdn_helo_hostname takes out
about 25% of all connections. Likewise, I reject any all-non-alpha
HELO with this simple check_helo_access regular expression:

!/[[:alpha:]]/                  550 5.5.2
         We find that all-numeric EHLO/HELO greetings are usually
         spam. If not, please ask your postmaster to correct the
         server's EHLO/HELO greeting.

When sensibly used (after or separate from user submission
restrictions), this catches a lot. (Around 2300 on my very small
site, 20ish users, in the past month or so.)

(Yes, my expression does block the technically valid HELO syntax of
bracketed IP address. This was a deliberate policy decision, and I
have not seen any legitimate MTA sending mail with such a HELO. I
figure, if you run a MTA that wants to send mail to mine, you ought
to have a domain name.)

Consider also upgrading to Postfix 2.8 if you're not already there,
and try the new triage daemon, postscreen(8):
     http://www.postfix.org/POSTSCREEN_README.html
It's in testing mode here, and I think it will do nicely to replace
my old reject_rbl_client restrictions.

> From time to time we're given addresses that should never
> be blocked but due to staff turnover & documentation not
> up-to-date, an address that should never be blocked was
> somehow blocked.
>
> Pardon me if this has been discussed before,
> what's the best way to go about preventing such mistakes?

Who is giving you these addresses? If these are user spam reports,
you're looking at the wrong part of the spam. Look at the IP
addresses from which they came.

> Is there a whitelist file that we can enter addresses that should
> never blocked so that even if this address is manually added
> into access_sender & access_recipient, they will still not be
> blocked (& possibly will be automatically removed from the
> two files access_sender/recipient).
>
> If there's such a whitelist file, presumably there should be 2
> of them, one for sending & receiving.  Let me know the full
> directory path & filename of the whitelist files

Again I think Brian covered that. These links might help:
     http://www.postfix.org/SMTPD_ACCESS_README.html
     http://jimsun.linxnet.com/misc/postfix-anti-UCE.txt
--
     Offlist mail to this address is discarded unless
     "/dev/rob0" or "not-spam" is in Subject: header

Le 31/01/2011 07:46, Daniel Bromberg a écrit :
> Hm, there must be a disconnect.
>
> I did read it, it sounded logical, I implemented it, and then my tests
> worked.
>
> I have:
>
> smtpd_sender_login_maps = mysql:/etc/postfix/mysql_sender_login_maps.cf
>
> smtpd_recipient_restrictions =
>    reject_sender_login_mismatch,
>    permit_mynetworks,
>    permit_sasl_authenticated,
> ...
>
> When I send use the wrong source name invalidorigin, I get this:
>
> *NOQUEUE: reject: RCPT from xxx <invalidorigin@...>: Sender
> address rejected: not owned by user validorigin@...>*
>
> But otherwise mail from the outside continues to come in to local
> (virtual) users fine, and using an authorized source name works.
>
> If I understand correctly, what it does during an unauthenticated
> session is that if there is a recognized virtual user in the MAIL FROM:
> field, it requires that the user be (SASL) logged in. If the MAIL FROM:
> is /not /a recognized virtual user, the rule does nothing and passes the
> filtering to the rest of the rules.

yes.

> This is naturally also what I want.

That was not my understanding. in your OP, you said:
>>> can only use the server to submit 'MAIL FROM:' their SASL
>>> authenticated username".

in the setup you did, users can send as ***@....


> All good no?
>
> Your final warning: "it won't prevent internal users from using an
> external sender address" -- define internal user? Those in my virtual
> table, or local Unix users? If the latter, I have none. As for "external
> sender address", are you referring to the envelope field, the Reply-to:
> field, or the From: field? If either of the latter two, yes we agreed
> earlier in the threat that that would have to be done with a cleanup
> filter.
>
> Clarify?

a virtual user authenticates as joe@... (which is his SASL
login) but sends as someone@... (where external.example may be
yahoo.com, hotmail.com, ... etc). I am talking about envelope sender here.

Le 01/02/2011 20:25, mouss a écrit :
> Le 31/01/2011 07:46, Daniel Bromberg a écrit :
>> Hm, there must be a disconnect.
>>
>> I did read it, it sounded logical, I implemented it, and then my tests
>> worked.
>>
>> I have:
>>
>> smtpd_sender_login_maps = mysql:/etc/postfix/mysql_sender_login_maps.cf
>>
>> smtpd_recipient_restrictions =
>>    reject_sender_login_mismatch,
>>    permit_mynetworks,
>>    permit_sasl_authenticated,
>> ...
>>
>> When I send use the wrong source name invalidorigin, I get this:
>>
>> *NOQUEUE: reject: RCPT from xxx <invalidorigin@...>: Sender
>> address rejected: not owned by user validorigin@...>*
>>
>> But otherwise mail from the outside continues to come in to local
>> (virtual) users fine, and using an authorized source name works.
>>
>> If I understand correctly, what it does during an unauthenticated
>> session is that if there is a recognized virtual user in the MAIL FROM:
>> field, it requires that the user be (SASL) logged in. If the MAIL FROM:
>> is /not /a recognized virtual user, the rule does nothing and passes the
>> filtering to the rest of the rules.
>
> yes.
>
>> This is naturally also what I want.
>
> That was not my understanding. in your OP, you said:
>>>> can only use the server to submit 'MAIL FROM:' their SASL
>>>> authenticated username".
>
> in the setup you did, users can send as ***@....
>
>
>> All good no?
>>
>> Your final warning: "it won't prevent internal users from using an
>> external sender address" -- define internal user? Those in my virtual
>> table, or local Unix users? If the latter, I have none. As for "external
>> sender address", are you referring to the envelope field, the Reply-to:
>> field, or the From: field? If either of the latter two, yes we agreed
>> earlier in the threat that that would have to be done with a cleanup
>> filter.
>>
>> Clarify?
>
> a virtual user authenticates as joe@... (which is his SASL
> login) but sends as someone@... (where external.example may be
> yahoo.com, hotmail.com, ... etc). I am talking about envelope sender here.

you can fix this in your submission/smtps:


submission_sender_restrictions =
    check_sender_access hash:/etc/postfix/submit_sender_domains
    reject

== submit_sender_domains
example.com OK
.example.com OK


This way, users of submission/smtp can only use an envelope sender of
the form *@... or *@*.example.com. and those you can control
with reject_sender_login_mismatch.

alternatively, you can simply return a dummy login for addresses not in
your domain when using sender_login_maps. but this is ugly (and requires
constructs like CASE WHEN/IF NULL in your sql query).

Wietse Venema:
> > >> 2.) Do AAAA/ip6.arpa lookup occur on client connects from IPv4
> > >> addresses or only when a client connects by IPv6 (name/reverse lookups)?
> > >
> > > The Postfix SMTP server's FCRDNS lookups use the getnameinfo() and
> > > getaddrinfo() system library functions. In dual-protocol mode,
> > > Postfix invokes getaddrinfo() with hints.ai_family=PF_UNSPEC; it
> > > would be smarter to pass the client's address family instead.
> > >
> > >  Wietse
> >
> > Is the last point still on todo list or considered worth to be
> > included in the future?
>
> It is not a high priority. Moreover, this means ripping up a
> low-level API, so it would take longer. I'm still fixing code
> that was merged into Postfix 9 years ago.

Fixed. Grumble. Another to hours of analysis, implementation, test,
documentation, and review.

	 Wietse

Le 31/01/2011 12:50, Andy Spiegl a écrit :
> [snip]
>> do you expand your aliases before a content_filter? ... etc.
> I still have a very basic postfix setup.

try after adding
dovecot_destination_recipient_limit = 1

if it still doesn't work, tell us how you defined the "dovecot"
transport? is it a pipe based transport in master.cf?

If so, take a look at the "COMMAND ATTRIBUTE SYNTAX" section in
	 http://www.postfix.org/pipe.8.html
check the "flags=...". excerpt:

               O      Prepend  an  "X-Original-To: recipient" mes-
                      sage header with the  recipient  address  as
                      given  to  Postfix.  Note: for this to work,
                      the    transport_destination_recipient_limit
                      must  be  1  (see  SINGLE-RECIPIENT DELIVERY
                      above for details).

                      This feature is available as of Postfix 2.0.

> SpamAssassin etc. will come
> later when I've solved the prefix issue...

when you do that, make sure you expand aliases after the filter (just
before delivery).


> [snip]

Le 01/02/2011 02:37, Daniel Bromberg a écrit :
> [snip]
> if ( /^X-Spam-Status: Yes/ )
> {
>     `test -d $HOME/Maildir/.Junk\ E-mail`
>     if ( $RETURNCODE != 0 )
>     {
>         `logger -i -p mail.info "maildroprc: creating .Junk E-mail in
> $HOME"`
>         `maildirmake $HOME/Maildir/.Junk\ E-mail`
>         `echo "Junk E-mail" >> $HOME/Maildir/subscriptions`
>         `chown vmail:mail $HOME/Maildir/subscriptions`
>         `chmod 700 $HOME/Maildir/subscriptions`
>     }
>     to "Maildir/.Junk E-mail"
> }


instead of `test ...`, use exceptions:

if (/^X-Spam-Flag:\s*YES/)
{
     exception    {
         to  "$_JUNK_DEST";
     }
     `maildirmake ... && chmod .... && chmod ... & echo ...`
     to  "$_JUNK_DEST";
}

this way, once the user has received a spam that caused the creation of
the maildir, the 'test' is no more performed.

you seem to use dovecot. then why use maildrop? can't you just use
dovecot-lda with sieve? (dovecot can auto-create maildirs).

Zitat von Wietse Venema <wietse@...>:

> Wietse Venema:
>> > >> 2.) Do AAAA/ip6.arpa lookup occur on client connects from IPv4
>> > >> addresses or only when a client connects by IPv6 (name/reverse
>> lookups)?
>> > >
>> > > The Postfix SMTP server's FCRDNS lookups use the getnameinfo() and
>> > > getaddrinfo() system library functions. In dual-protocol mode,
>> > > Postfix invokes getaddrinfo() with hints.ai_family=PF_UNSPEC; it
>> > > would be smarter to pass the client's address family instead.
>> > >
>> > >  Wietse
>> >
>> > Is the last point still on todo list or considered worth to be
>> > included in the future?
>>
>> It is not a high priority. Moreover, this means ripping up a
>> low-level API, so it would take longer. I'm still fixing code
>> that was merged into Postfix 9 years ago.
>
> Fixed. Grumble. Another to hours of analysis, implementation, test,
> documentation, and review.
>
>  Wietse

Sorry, i did not want to bother you. My only intention was to prevent
that something maybe valuable for performance might get lost.

Many Thanks

Andreas

> instead of `test ...`, use exceptions:
>
> if (/^X-Spam-Flag:\s*YES/)
> {
>      exception    {
>          to  "$_JUNK_DEST";
>      }
>      `maildirmake ...&&  chmod ....&&  chmod ...&  echo ...`
>      to  "$_JUNK_DEST";
> }
>
> this way, once the user has received a spam that caused the creation of
> the maildir, the 'test' is no more performed.

Yes, thanks. I glanced at exceptions but was just glad to have something
working.

> you seem to use dovecot. then why use maildrop? can't you just use
> dovecot-lda with sieve? (dovecot can auto-create maildirs).
Finally, the right piece. I looked for a while for tools to work with
Spamassassin and only found procmail and maildrop. I didn't think to
explore Dovecot's secondary tools because conceptually, I had it boxed
in as an IMAP service only, and not the final link in the SMTP chain.

It felt wrong to have to run the Cyrus authentication daemon and a
conceptually duplicate SQL config file just to help maildrop talk to my
database.

-Daniel

>> in the setup you did, users can send as ***@....
>>
>>
>>> All good no?
>>>
>>> Your final warning: "it won't prevent internal users from using an
>>> external sender address" -- define internal user? Those in my virtual
>>> table, or local Unix users? If the latter, I have none. As for "external
>>> sender address", are you referring to the envelope field, the Reply-to:
>>> field, or the From: field? If either of the latter two, yes we agreed
>>> earlier in the threat that that would have to be done with a cleanup
>>> filter.
>>>
>>> Clarify?
>> a virtual user authenticates as joe@... (which is his SASL
>> login) but sends as someone@... (where external.example may be
>> yahoo.com, hotmail.com, ... etc). I am talking about envelope sender here.

Still a disconnect compared to what I am seeing. When I re-configure my
MUA to use 'somebody@...' as the Sender to send to
anyone@..., and SASL authenticate as authuser@... to
the submission port, Postfix replies:

"An error occurred while sending mail. The mail server responded:  5.7.1
<somebody@...>: Sender address rejected: not owned by user
authuser@.... Please check the message recipient
anyone@... and try again."

This is without the additional check_sender_access you describe as needed.
As quick re-cap, I have:

submission_client_restrictions =
     reject_sender_login_mismatch,
     permit_sasl_authenticated,
     reject

AND:

smtp.example.com:smtps     inet  n       -       n       -       -      smtpd
    -o smtpd_tls_wrappermode=yes
    -o smtpd_sasl_auth_enable=yes
    -o smtpd_client_restrictions=$submission_client_restrictions
    -o syslog_name=postfix-submission

Is there some other part of the config I haven't discussed and need to, that is
making this work already for me?

-Daniel

We are using postfix with debian lenny... 

We are receiving what appears to be backscatter from spam that is using a valid address in the Return Path. I have included an example of the header info from one of the spam messages below. The “From” and “To” addresses just seem to be random and are not related to us in any way. Does anyone know to block this sort of backscatter?  

Original message headers:

Return-Path: <soa@**[ourdomain.actual.domain]**>
Received: from 195-191-72-102.optolan.net.ua (unknown [195.191.72.102])
                by smtp-0.counselschambers.com.au (Postfix) with ESMTP id 1D400396B7E
                for <soars@...>; Wed,  2 Feb 2011 08:28:43 +1100 (EST)
From: no-reply811@...
To: <soars@...>
Subject: Position opening in your area
MIME-Version: 1.0
Importance: High
Content-Type: text/html
Message-ID: <20110201212844.1D400396B7E@...>
Date: Wed, 2 Feb 2011 08:28:43 +1100

Thanks

Simon

Simon:
> We are using postfix with debian lenny...
>
>
> We are receiving what appears to be backscatter from spam that is using a
> valid address in the Return Path. I have included an example of the header
> info from one of the spam messages below. The _From_ and _To_ addresses just
> seem to be random and are not related to us in any way. Does anyone know to
> block this sort of backscatter?
>
>
> Original message headers:

Safe suggestion: if there is any information in the header or body
content that appears to be common between spam messages, then you
can use a header_checks or body_checks HOLD action and freeze the
mail in the queue, then clean it up later.

Not-so-safe suggestion: defer all bounces for the affected address.

Untested example:

/etc/postfix/main.cf
     restriction_classes = defer-bounce
     defer-bounce = check_sender_access hash:/etc/postfix/mail_access
     smtpd_recipient_restrictions =
	 permit_mynetworks
	 ...
	 reject_unauth_destination
	 check_recipient_access hash:/etc/postfix/rcpt_access
	 ...

/etc/postfix/rcpt_access:
     victim@... defer-bounce

/etc/postfix/mail_access:
     <> 		 defer this recipient is receiving too many bounces
     mailer-daemon@ defer this recipient is receiving too many bounces
     postmaster@  defer this recipient is receiving too many bounces

	 Wietse

>
>
> Return-Path: <soa@* <soa@...>*[ourdomain.actual.domain]**>
> Received: from 195-191-72-102.optolan.net.ua (unknown [195.191.72.102])
>                 by smtp-0.counselschambers.com.au (Postfix) with ESMTP id
> 1D400396B7E
>                 for <soars@...>; Wed,  2 Feb 2011 08:28:43 +1100
> (EST)
> From: no-reply811@...
> To: <soars@...>
> Subject: Position opening in your area
> MIME-Version: 1.0
> Importance: High
> Content-Type: text/html
> Message-ID: <20110201212844.1D400396B7E@...>
> Date: Wed, 2 Feb 2011 08:28:43 +1100
>
> Thanks
>
> Simon

We are using postfix with debian lenny... 

We are receiving what appears to be backscatter from spam that is using a valid address in the Return Path. I have included an example of the header info from one of the spam messages below. The “From” and “To” addresses just seem to be random and are not related to us in any way. Does anyone know to block this sort of backscatter?  

Original message headers:

 

Return-Path: <soa@**[ourdomain.actual.domain]**>
Received: from 195-191-72-102.optolan.net.ua (unknown [195.191.72.102])
                by smtp-0.counselschambers.com.au (Postfix) with ESMTP id 1D400396B7E
                for <soars@...>; Wed,  2 Feb 2011 08:28:43 +1100 (EST)
From: no-reply811@...
To: <soars@...>
Subject: Position opening in your area
MIME-Version: 1.0
Importance: High
Content-Type: text/html
Message-ID: <20110201212844.1D400396B7E@...>
Date: Wed, 2 Feb 2011 08:28:43 +1100

Thanks

Simon

On 2011-02-02 Simon wrote:
> We are receiving what appears to be backscatter from spam that is
> using a valid address in the Return Path. I have included an example
> of the header info from one of the spam messages below. The ?From? and
> ?To? addresses just seem to be random and are not related to us in any
> way. Does anyone know to block this sort of backscatter?

I wrote a backscatter filter based on smtpprox to handle this.

http://www.planetcobalt.net/sdb/backscatter.shtml

WFM, but AFAIK not tested outside low traffic environments.

Regards
Ansgar Wiechers
--
"Abstractions save us time working, but they don't save us time learning."
--Joel Spolsky