How do we install Fail2Ban?

[Terra]

A YAML file is a simplified object notation file, in that I can read in the file directly into my program's config data structures and validate the input for sanity... Then I would use your config data as the criteria to do the blocking... I would invite you to look at the Wikipedia pages for both JSON and YAML, as you have the proclivity to learn new things...

YAML is basically the config file you would use if you plan to edit it manually
JSON is usually a better fit for a config file that is handled directly by a program

YAML and JSON are usually interchangeable in that they are both textual object representations, one is more for humans and the other is more for programs to store (marshalling) their configuration data or communicate object data to other programs...

In short, you would create the YAML file in a text editor, paying attention to the spacing, and uploading it in Unix mode to ensure it only contains LF (LineFeed) terminators... This YAML or JSON file is just an intermediate step until the configuration controls are placed into the CNC and we'll store the config files automatically...

As far as the configuration, what the default will cause the scanner to do is:
1) scan your log file data in realtime
2) record each 404 log lines IP address
3) if same IP has more than 3 (maxretry) 404's within a 600 (findtime) second time window, then firewall (REJECT) their IP for 600 (bantime) seconds

Given this will be written for scale, I will probably write this in the Go language (golang) as it is a natural fit for running many watchers in concurrency and being crazy fast about it in the process...

As it stands now, I'm thinking about the implementation, and I may tie this into another subsystem I'm writing for inline log file rotations... I'm rewriting it from scratch because I wasn't happy with its processing overhead, not to mention it didn't handle the leap second properly which led to a brief amount of schizophrenia on our testing sites... Going this route, I can dispense completely with the inotify complexity and attain K.I.S.S. realtime operational properties... Once the data structures are mapped out, the actual implementation will become less in flux and the direction it wants to go will make itself (organically) apparent...

[martian101]

In short, you would create the YAML file in a text editor, paying attention to the spacing, and uploading it in Unix mode to ensure it only contains LF (LineFeed) terminators...

Thanks for this, Terra. Okay, now Just to verify, these are what I did:

1. Downloaded Notepad++ (as I see that I can't create a YAML file using Notepad only).

2. Placed the code:

FQdenier:
email: "user@domain.com"
status:
- code: 404
maxretry: 3
findtime: 600
bantime: 600

3. Saved it as .yaml file. I also checked the Encoding and I saw that it was in "Encode in UTF-8." I don't know if that's the Unix mode.

So far, that's all. Did I do them right?

If yes, my question then is where on my CNC should I place the .yaml file? Outside /www directory or inside? And where in the /www directory?

[martian101]

xdom/FQdenier.yaml

I reviewed the previous post and saw this. I guess this is outside /www directory.

The extension of my YAML file is .yml and not .yaml. I don't know if they're the same. I tried looking for an exact .yaml extension in Notepad++ but I can't find any

[Terra]

I am extending the file extension list to look for:
xdom/FQdenier.{yml,yaml,json}

in that order, the first one found will be used...

Also, I copied your existing FQdenier.yml ==> FQdenier.yml-SAVE and made corrections to FQdenier.yml... Feel free to compare the difference in spacing and syntax...

Otherwise, the meat of the configuration looks good...

A nice little resource to convert YAML/JSON, and will also validate either can be found at: https://www.json2yaml.com/

[martian101]

@Terra: Thanks again for the help.

Also, I copied your existing FQdenier.yml ==> FQdenier.yml-SAVE and made corrections to FQdenier.yml

Yes, I noticed there was a difference in the spacing as well as the symbol "-" being added. Thank you for the tip.

nice little resource to convert YAML/JSON, and will also validate either can be found at: https://www.json2yaml.com/

Checked the code you taught and it was green. Again, thanks!

I would apply this to my second website and upcoming third (will host it again on FutureQuest - one satisfied customer here!). I'll just save the other .yaml for reference.

Thanks again and more power for FutureQuest!

[martian101]

@Terra:

I was thinking of expanding the defense against malicious 404s. And I was experimenting with the YAML code you taught.

The idea is to increase the ban time of IP addresses that make large 404 events in just a short time, say, if an IP has made 9248 404s in just 1 second, I want to ban that IP longer.

I don't know if such technology exists, though I know about brute-force attacks. I don't know if brute-force attacks on URLs exist.

But anyway, here's the code:

FQdenier:
email: (username@example.com)
status:
- code: 404
maxretry: 6
findtime: 666
bantime: 604800
- code: 404
maxretry: 12
findtime: 333
bantime: 1209600
- code: 404
maxretry: 36
findtime: 167
bantime: 2419200
- code: 404
maxretry: 72
findtime: 84
bantime: 4838400
- code: 404
maxretry: 144
findtime: 42
bantime: 9676800
- code: 404
maxretry: 288
findtime: 21
bantime: 19353600
- code: 404
maxretry: 576
findtime: 11
bantime: 38707200
- code: 404
maxretry: 1152
findtime: 6
bantime: 77414400
- code: 404
maxretry: 2312
findtime: 3
bantime: 154828800
- code: 404
maxretry: 4624
findtime: 2
bantime: 309657600
- code: 404
maxretry: 9248
findtime: 1
bantime: 619315200

Do you think this makes sense?

I checked it with https://www.json2yaml.com/convert-yaml-to-json and, so far, it's green.

[Terra]

Having it tiered like that won't really work because this is being blocked at the firewall and Apache will never see it therefore the scanner will not be able to process it any further until the bantime expires and Apache logs the accesses once again... Basically, your strictest findtime+maxretry would win...

In short, there is no real way to accomplish these new parameters on the same status code...

[martian101]

Oh, okay.

Thank you, Terra