jMilter - JSON mail filtering and manipulation protocol

jmilter_request.md

Sample jMilter request for the DATA stage:

{
    "context": {
        "stage": "DATA",
        "sasl": {
            "login": "user",
            "method": "plain"
        },
        "client": {
            "ip": "192.168.1.1",
            "port": 34567,
            "ptr": "mail.example.com",
            "ehlo": "mail.example.com",
            "activeConnections": 1
        },
        "tls": {
            "version": "1.3",
            "cipher": "TLS_AES_256_GCM_SHA384",
            "cipherBits": 256,
            "certIssuer": "Let's Encrypt",
            "certSubject": "mail.example.com"
        },
        "server": {
            "name": "Stalwart Mail Server",
            "port": 25,
            "ip": "192.168.2.2"
        },
        "queue": {
            "id": "1234567890"
        },
        "protocol": {
            "version": "1.0"
        }
    },
    "envelope": {
        "from": {
            "address": "[email protected]",
            "parameters": {
                "size": 12345
            }
        },
        "to": [
            {
                "address": "[email protected]",
                "parameters": {
                    "orcpt": "rfc822; [email protected]"
                }
            },
            {
                "address": "[email protected]",
                "parameters": null
            }
        ]
    },
    "message": {
        "headers": [
            [
                "From",
                "John Doe <[email protected]>"
            ],
            [
                "To",
                "Bill <[email protected]>, Jane <[email protected]>"
            ],
            [
                "Subject",
                "Hello, World!"
            ]
        ],
        "serverHeaders": [
            [
                "Received",
                "from mail.example.com (mail.example.com [192.168.1.1]) by mail.foobar.com (Stalwart Mail Server) with ESMTPS id 1234567890"
            ]
        ],
        "contents": "Hello, World!\r\n",
        "size": 12345
    }
}

Notes:

• HTTP POST requests are used with authentication being optional.
• Message bodies are provided raw as content filters usually need to analyse the MIME structure as well.
• jMilters can be called from any stage of the SMTP transaction so most of these fields are optional.
• message.headers[][] is an array of arrays to keep the JSON representation compact. This can be changed to an array of objects if needed.

jmilter_response.md

Sample jMilter response:

{
    "action": "accept",
    "response": {
        "status": 250,
        "enhancedStatus": "2.0.0",
        "message": "Message accepted",
        "disconnect": false
    },
    "modifications": [
        {
            "type": "changeFrom",
            "value": "[email protected]",
            "parameters": {
                "size": 54321
            }
        },
        {
            "type": "addRcpt",
            "value": "[email protected]",
            "parameters": null
        },
        {
            "type": "deleteRcpt",
            "value": "[email protected]"
        },
        {
            "type": "replaceBody",
            "value": "This is the new body\r\n"
        },
        {
            "type": "addHeader",
            "name": "X-Spam-Status",
            "value": "No"
        },
        {
            "type": "insertHeader",
            "index": 1,
            "name": "X-Filtered-By",
            "value": "Custom Filter v1.1",
        },
        {
            "type": "changeHeader",
            "index": 4,
            "name": "Subject",
            "value": "This is the new subject",
        },
        {
            "type": "deleteHeader",
            "index": 1,
            "name": "X-Mailer",
        }
    ]
}

Notes:

• response is optional and allows the filter to customise the SMTP response.
• Possible actions are:
  • accept
  • discard
  • reject
  • quarantine

Nice. WAY easier to understand than the original milter...

Little question on my side of how the multiparts are to be represented in the body as json?
Also similar to JMAP going out of band for large content might be helpful?
Also the way encoding of string is managed needs to be clarified (raw header, so with Q-encoding?).
I would also document the HTTP semantic (VERB, URL, status codes, etc..).

Little question on my side of how the multiparts are to be represented in the body as json?

The message body needs to be provided raw as spam filters usually need to analyse the MIME structure of the message as well. The headers are parsed but not decoded.

Also the way encoding of string is managed needs to be clarified (raw header, so with Q-encoding?). I would also document the HTTP semantic (VERB, URL, status codes, etc..).

I agree, this first iteration is just to get feedback on the JSON schema mostly.

What does it mean to replaceBody? A plain body only or to provide all body alternatives? It is not clear to me.

Here is a use case example. Imagine a feature 'URL filtering', where all URLs are replaced with an encrypted URL (used like hashes) to the service (and their real values are stored).
On click from an MUA, those encrypted URLs are decrypted, checked with APIs like Google Safe Browsing, and redirected. And there is a need to be replaced both plain and HTML bodies.

What does it mean to replaceBody? A plain body only or to provide all body alternatives? It is not clear to me.

It replaces the entire message contents except the headers. The name comes from milter but it should be renamed to replaceContents to avoid confusion.

This also needs a clarification of what it means.

{
	"type": "quarantine",
	"value": "We have quarantined this message for further review."
}

To me, this sounds like queue placement? quarantine/defer/incoming.
But then why are both "changeHeader" and "quarantine" types, for example?

What does it mean to replaceBody? A plain body only or to provide all body alternatives? It is not clear to me.

It replaces the entire message contents except the headers. The name comes from milter but it should be renamed to replaceContents to avoid confusion.

replaceContents make sense.

To me, this sounds like queue placement? quarantine/defer/incoming.

Yes, it moves the message to the quarantine queue. Quarantining messages is not something that Stalwart supports for it is part of the Milter protocol.

But then why are both "changeHeader" and "quarantine" types, for example?

Milter includes quarantine as one of the possible modifications, I guess this is to allow combining quarantining with any of the possible actions. Perhaps instead of having it as an action we can add an optional quarantine boolean to the response, something like this:

{
    "action": "accept",
    "quarantine": true,
    "modifications": []
}

Edit: Also quarantine could be an action with some additional parameters to either accept, reject, etc.

replaceContents make sense.

Just updated the schema. Also renamed body to contents.