Tuesday, August 1, 2017

Python - 'Hello World!'

I decided to learn Python a bit but I prefer real problems not 'courses' way of learning as I'm too fast bored. Hopefully I worked on one issue where VPN (site-to-site) worked very unstable. Sometimes everything was fine, sometimes some hosts couldn't pass through. I discovered that the problem is related with overlapping encryption domains. There were some 'host' and 'network' entries which were added over the time when the project went from development to production phase. The only one problem was the old entries were not removed. Once all overlapping entries were removed everything works fine. I tried to find a way how to check configuration for all remaining tunnels but not to spend too much time on it. So I thought about Python then...

This is the configuration part which I want to work on:


object network 172.16.0.0_16
 subnet 172.16.0.0 255.255.0.0
object network 192.168.1.0_24
 subnet 192.168.1.0 255.255.254.0
object network 192.168.1.0_23
 subnet 192.168.1.0 255.255.254.0
object network 172.16.250.5
 host 172.16.250.5
object network 172.16.250.0_24
 subnet 172.16.250.0 255.255.255.0
object network 172.16.250.5_
 host 172.16.250.5
object network 172.16.250.110
 host 172.16.250.110
object network 10.10.0.100_h
 host 10.10.0.100
object-group network TEST1
 network-object object 172.16.0.0_16
 network-object object 172.16.250.5
 network-object object 172.16.250.0_24
 network-object object 172.16.250.5_
 network-object object 172.16.250.110
 network-object 172.16.250.0 255.255.255.0
object-group network TEST2
 network-object host 10.10.0.100
 network-object host 10.10.0.20
 network-object host 172.16.250.5
 network-object 192.168.1.0 255.255.255.0
 network-object object 192.168.1.0_24
 network-object object 192.168.1.0_23
 network-object object 10.10.0.100_h
 
 
 
For those who are not familiar with access lists (named) this is a short explanation:
 
 
# object-group network OBJECT-GROUP-NAME    <-- parent
# |-network-object host IP
# |-network-object IP Mask
# |-network-object object OBJECT-NAME
# |-group-object GROUP-OBJECT-NAME <-- not supported yet
#
#
# object network OBJECT-NAME                <-- child
# |-subnet IP MASK
# |-host IP
# |-range IP IP <-- not supported yet
# |-fqdn <-- not supported yet
 
 
So, under ‘object-group’ you can define ‘host’, ‘network’, ‘object’ which calls another object definition (object network) and ‘group object’ which calls ‘object-group’. As you see there are many ways to add overlapping elements. The configuration is not too big so I can find duplicates or overlapping entries manually.
 
 
 
object-group network TEST1
 network-object object 172.16.0.0_16   --> subnet 172.16.0.0 255.255.0.0
 network-object object 172.16.250.5    --> host 172.16.250.5
 network-object object 172.16.250.0_24 --> subnet 172.16.250.0 255.255.255.0
 network-object object 172.16.250.5_   --> host 172.16.250.5
 network-object object 172.16.250.110  --> host 172.16.250.110
 network-object 172.16.250.0 255.255.255.0
 
 
In object-group network ‘TEST1’ we have 2 the same hosts (defined in different way):
 
 
host 172.16.250.5

network statement which overlaps all hosts entries:

172.16.250.0 255.255.255.0

and subnet with /16 mask which overlaps all items:
 
172.16.0.0 255.255.0.0

So, this one is required only as rest of the entries are overlapped by this one:
 
172.16.0.0/16

The same analysis I do for next object:

object-group network TEST2
 network-object host 10.10.0.100
 network-object host 10.10.0.20
 network-object host 172.16.250.5
 network-object 192.168.1.0 255.255.255.0
 network-object object 192.168.1.0_24 --> subnet 192.168.1.0 255.255.254.0
 network-object object 192.168.1.0_23 --> subnet 192.168.1.0 255.255.254.0
 network-object object 10.10.0.100_h --> host 10.10.0.100
 
As you probably noticed there is typo in the object name:
 
network-object object 192.168.1.0_24 --> subnet 192.168.1.0 255.255.254.0

in the object name definition there is ‘_24’ but in the object subnet has /23 mask. I did it to present this kind of human errors which may be found by my script. In this object-group there are following duplicates:

network-object host 10.10.0.100 
network-object object 10.10.0.100_h --> host 10.10.0.100
 
then
 
network-object object 192.168.1.0_24 --> subnet 192.168.1.0 255.255.254.0
network-object object 192.168.1.0_23 --> subnet 192.168.1.0 255.255.254.0
 

and
 
network-object 192.168.1.0 255.255.255.0

is overlapped by 192.168.1.0/23.

In a production network such configuration may have 1000+ lines and it is impossible to analyze them manually.

I wrote a script ‘EDO’ – encryption domain optimizer. I know it can be written in much better, elegant, etc… way but this is my first one and I’m going to work on it. There are some not-supported features.

This is the output I get:


[hzw@zeus edo]$ python edo.py acl-test1.txt

final version of host duplicates:
['TEST1', '172.16.250.5', '255.255.255.255']
['TEST2', '10.10.0.100', '255.255.255.255']

final version of network duplicates:
['TEST1', '172.16.250.0', '255.255.255.0']
['TEST2', '192.168.1.0', '255.255.254.0']

final version of network/subnet duplicates:
['TEST1', '172.16.250.5', '255.255.255.255']
['TEST1', '172.16.250.0', '255.255.255.0']
-----
['TEST1', '172.16.250.5', '255.255.255.255']
['TEST1', '172.16.0.0', '255.255.0.0']
-----
['TEST1', '172.16.250.110', '255.255.255.255']
['TEST1', '172.16.250.0', '255.255.255.0']
-----
['TEST1', '172.16.250.110', '255.255.255.255']
['TEST1', '172.16.0.0', '255.255.0.0']
-----
['TEST1', '172.16.250.0', '255.255.255.0']
['TEST1', '172.16.250.5', '255.255.255.255']
-----
['TEST1', '172.16.250.0', '255.255.255.0']
['TEST1', '172.16.250.110', '255.255.255.255']
-----
['TEST1', '172.16.250.0', '255.255.255.0']
['TEST1', '172.16.0.0', '255.255.0.0']
-----
['TEST2', '192.168.1.0', '255.255.255.0']
['TEST2', '192.168.1.0', '255.255.254.0']
-----
['TEST1', '172.16.0.0', '255.255.0.0']
['TEST1', '172.16.250.5', '255.255.255.255']
-----
['TEST1', '172.16.0.0', '255.255.0.0']
['TEST1', '172.16.250.110', '255.255.255.255']
-----
['TEST1', '172.16.0.0', '255.255.0.0']
['TEST1', '172.16.250.0', '255.255.255.0']
-----
['TEST2', '192.168.1.0', '255.255.254.0']
['TEST2', '192.168.1.0', '255.255.255.0']
-----
[hzw@zeus edo]$


There is also ‘debug’ version which provide more details:


[hzw@zeus edo]$ python edo.py acl-test1.txt -d

Object network table:
[2, '172.16.0.0_16', 'subnet', '172.16.0.0 255.255.0.0', '172.16.0.0 255.255.0.0']
[4, '192.168.1.0_24', 'subnet', '192.168.1.0 255.255.254.0', '192.168.1.0 255.255.254.0']
[6, '192.168.1.0_23', 'subnet', '192.168.1.0 255.255.254.0', '192.168.1.0 255.255.254.0']
[8, '172.16.250.5', 'host', '172.16.250.5', '255.255.255.255']
[10, '172.16.250.0_24', 'subnet', '172.16.250.0 255.255.255.0', '172.16.250.0 255.255.255.0']
[12, '172.16.250.5_', 'host', '172.16.250.5', '255.255.255.255']
[14, '172.16.250.110', 'host', '172.16.250.110', '255.255.255.255']
[16, '10.10.0.100_h', 'host', '10.10.0.100', '255.255.255.255']

Object group network table:
[18, 'TEST1', 'object', '172.16.0.0_16', '172.16.0.0_16']
[19, 'TEST1', 'object', '172.16.250.5', '172.16.250.5']
[20, 'TEST1', 'object', '172.16.250.0_24', '172.16.250.0_24']
[21, 'TEST1', 'object', '172.16.250.5_', '172.16.250.5_']
[22, 'TEST1', 'object', '172.16.250.110', '172.16.250.110']
[23, 'TEST1', 'network', '172.16.250.0', '255.255.255.0']
[25, 'TEST2', 'host', '10.10.0.100', '255.255.255.255']
[26, 'TEST2', 'host', '10.10.0.20', '255.255.255.255']
[27, 'TEST2', 'host', '172.16.250.5', '255.255.255.255']
[28, 'TEST2', 'network', '192.168.1.0', '255.255.255.0']
[29, 'TEST2', 'object', '192.168.1.0_24', '192.168.1.0_24']
[30, 'TEST2', 'object', '192.168.1.0_23', '192.168.1.0_23']
[31, 'TEST2', 'object', '10.10.0.100_h', '10.10.0.100_h']

Object group network table:
[23, 'TEST1', 'network', '172.16.250.0', '255.255.255.0']
[25, 'TEST2', 'host', '10.10.0.100', '255.255.255.255']
[26, 'TEST2', 'host', '10.10.0.20', '255.255.255.255']
[27, 'TEST2', 'host', '172.16.250.5', '255.255.255.255']
[28, 'TEST2', 'network', '192.168.1.0', '255.255.255.0']
[18, 'TEST1', 'subnet', '172.16.0.0', '255.255.0.0']
[19, 'TEST1', 'host', '172.16.250.5', '255.255.255.255']
[20, 'TEST1', 'subnet', '172.16.250.0', '255.255.255.0']
[21, 'TEST1', 'host', '172.16.250.5', '255.255.255.255']
[22, 'TEST1', 'host', '172.16.250.110', '255.255.255.255']
[29, 'TEST2', 'subnet', '192.168.1.0', '255.255.254.0']
[30, 'TEST2', 'subnet', '192.168.1.0', '255.255.254.0']
[31, 'TEST2', 'host', '10.10.0.100', '255.255.255.255']

['00001010000010100000000001100100', 'TEST2', '11111111111111111111111111111111', 32, 25, 'host', '10.10.0.100', '255.255.255.255']
['00001010000010100000000000010100', 'TEST2', '11111111111111111111111111111111', 32, 26, 'host', '10.10.0.20', '255.255.255.255']
['10101100000100001111101000000101', 'TEST2', '11111111111111111111111111111111', 32, 27, 'host', '172.16.250.5', '255.255.255.255']
['10101100000100001111101000000101', 'TEST1', '11111111111111111111111111111111', 32, 19, 'host', '172.16.250.5', '255.255.255.255']
['10101100000100001111101000000101', 'TEST1', '11111111111111111111111111111111', 32, 21, 'host', '172.16.250.5', '255.255.255.255']
['10101100000100001111101001101110', 'TEST1', '11111111111111111111111111111111', 32, 22, 'host', '172.16.250.110', '255.255.255.255']
['00001010000010100000000001100100', 'TEST2', '11111111111111111111111111111111', 32, 31, 'host', '10.10.0.100', '255.255.255.255']

['10101100000100001111101000000000', 'TEST1', '11111111111111111111111100000000', 24, 23, 'network', '172.16.250.0', '255.255.255.0']
['11000000101010000000000100000000', 'TEST2', '11111111111111111111111100000000', 24, 28, 'network', '192.168.1.0', '255.255.255.0']
['10101100000100000000000000000000', 'TEST1', '11111111111111110000000000000000', 16, 18, 'subnet', '172.16.0.0', '255.255.0.0']
['10101100000100001111101000000000', 'TEST1', '11111111111111111111111100000000', 24, 20, 'subnet', '172.16.250.0', '255.255.255.0']
['11000000101010000000000100000000', 'TEST2', '11111111111111111111111000000000', 23, 29, 'subnet', '192.168.1.0', '255.255.254.0']
['11000000101010000000000100000000', 'TEST2', '11111111111111111111111000000000', 23, 30, 'subnet', '192.168.1.0', '255.255.254.0']

final version of host duplicates:
['TEST1', '172.16.250.5', '255.255.255.255']
['TEST2', '10.10.0.100', '255.255.255.255']

final version of network duplicates:
['TEST1', '172.16.250.0', '255.255.255.0']
['TEST2', '192.168.1.0', '255.255.254.0']

Merged host and subnet table (unique)
['00001010000010100000000001100100', 'TEST2', '11111111111111111111111111111111', 32, 25, 'host', '10.10.0.100', '255.255.255.255']
['00001010000010100000000000010100', 'TEST2', '11111111111111111111111111111111', 32, 26, 'host', '10.10.0.20', '255.255.255.255']
['10101100000100001111101000000101', 'TEST2', '11111111111111111111111111111111', 32, 27, 'host', '172.16.250.5', '255.255.255.255']
['10101100000100001111101000000101', 'TEST1', '11111111111111111111111111111111', 32, 19, 'host', '172.16.250.5', '255.255.255.255']
['10101100000100001111101001101110', 'TEST1', '11111111111111111111111111111111', 32, 22, 'host', '172.16.250.110', '255.255.255.255']
['10101100000100001111101000000000', 'TEST1', '11111111111111111111111100000000', 24, 23, 'network', '172.16.250.0', '255.255.255.0']
['11000000101010000000000100000000', 'TEST2', '11111111111111111111111100000000', 24, 28, 'network', '192.168.1.0', '255.255.255.0']
['10101100000100000000000000000000', 'TEST1', '11111111111111110000000000000000', 16, 18, 'subnet', '172.16.0.0', '255.255.0.0']
['11000000101010000000000100000000', 'TEST2', '11111111111111111111111000000000', 23, 29, 'subnet', '192.168.1.0', '255.255.254.0']

-------------------------
i['10101100000100001111101000000101', 'TEST1', '11111111111111111111111111111111', 32, 19, 'host', '172.16.250.5', '255.255.255.255']
j['10101100000100001111101000000000', 'TEST1', '11111111111111111111111100000000', 24, 23, 'network', '172.16.250.0', '255.255.255.0']
-------------------------
i['10101100000100001111101000000101', 'TEST1', '11111111111111111111111111111111', 32, 19, 'host', '172.16.250.5', '255.255.255.255']
j['10101100000100000000000000000000', 'TEST1', '11111111111111110000000000000000', 16, 18, 'subnet', '172.16.0.0', '255.255.0.0']
-------------------------
i['10101100000100001111101001101110', 'TEST1', '11111111111111111111111111111111', 32, 22, 'host', '172.16.250.110', '255.255.255.255']
j['10101100000100001111101000000000', 'TEST1', '11111111111111111111111100000000', 24, 23, 'network', '172.16.250.0', '255.255.255.0']
-------------------------
i['10101100000100001111101001101110', 'TEST1', '11111111111111111111111111111111', 32, 22, 'host', '172.16.250.110', '255.255.255.255']
j['10101100000100000000000000000000', 'TEST1', '11111111111111110000000000000000', 16, 18, 'subnet', '172.16.0.0', '255.255.0.0']
-------------------------
i['10101100000100001111101000000000', 'TEST1', '11111111111111111111111100000000', 24, 23, 'network', '172.16.250.0', '255.255.255.0']
j['10101100000100001111101000000101', 'TEST1', '11111111111111111111111111111111', 32, 19, 'host', '172.16.250.5', '255.255.255.255']
-------------------------
i['10101100000100001111101000000000', 'TEST1', '11111111111111111111111100000000', 24, 23, 'network', '172.16.250.0', '255.255.255.0']
j['10101100000100001111101001101110', 'TEST1', '11111111111111111111111111111111', 32, 22, 'host', '172.16.250.110', '255.255.255.255']
-------------------------
i['10101100000100001111101000000000', 'TEST1', '11111111111111111111111100000000', 24, 23, 'network', '172.16.250.0', '255.255.255.0']
j['10101100000100000000000000000000', 'TEST1', '11111111111111110000000000000000', 16, 18, 'subnet', '172.16.0.0', '255.255.0.0']
-------------------------
i['11000000101010000000000100000000', 'TEST2', '11111111111111111111111100000000', 24, 28, 'network', '192.168.1.0', '255.255.255.0']
j['11000000101010000000000100000000', 'TEST2', '11111111111111111111111000000000', 23, 29, 'subnet', '192.168.1.0', '255.255.254.0']
-------------------------
i['10101100000100000000000000000000', 'TEST1', '11111111111111110000000000000000', 16, 18, 'subnet', '172.16.0.0', '255.255.0.0']
j['10101100000100001111101000000101', 'TEST1', '11111111111111111111111111111111', 32, 19, 'host', '172.16.250.5', '255.255.255.255']
-------------------------
i['10101100000100000000000000000000', 'TEST1', '11111111111111110000000000000000', 16, 18, 'subnet', '172.16.0.0', '255.255.0.0']
j['10101100000100001111101001101110', 'TEST1', '11111111111111111111111111111111', 32, 22, 'host', '172.16.250.110', '255.255.255.255']
-------------------------
i['10101100000100000000000000000000', 'TEST1', '11111111111111110000000000000000', 16, 18, 'subnet', '172.16.0.0', '255.255.0.0']
j['10101100000100001111101000000000', 'TEST1', '11111111111111111111111100000000', 24, 23, 'network', '172.16.250.0', '255.255.255.0']
-------------------------
i['11000000101010000000000100000000', 'TEST2', '11111111111111111111111000000000', 23, 29, 'subnet', '192.168.1.0', '255.255.254.0']
j['11000000101010000000000100000000', 'TEST2', '11111111111111111111111100000000', 24, 28, 'network', '192.168.1.0', '255.255.255.0']

final version of network/subnet duplicates:
['TEST1', '172.16.250.5', '255.255.255.255']
['TEST1', '172.16.250.0', '255.255.255.0']
-----
['TEST1', '172.16.250.5', '255.255.255.255']
['TEST1', '172.16.0.0', '255.255.0.0']
-----
['TEST1', '172.16.250.110', '255.255.255.255']
['TEST1', '172.16.250.0', '255.255.255.0']
-----
['TEST1', '172.16.250.110', '255.255.255.255']
['TEST1', '172.16.0.0', '255.255.0.0']
-----
['TEST1', '172.16.250.0', '255.255.255.0']
['TEST1', '172.16.250.5', '255.255.255.255']
-----
['TEST1', '172.16.250.0', '255.255.255.0']
['TEST1', '172.16.250.110', '255.255.255.255']
-----
['TEST1', '172.16.250.0', '255.255.255.0']
['TEST1', '172.16.0.0', '255.255.0.0']
-----
['TEST2', '192.168.1.0', '255.255.255.0']
['TEST2', '192.168.1.0', '255.255.254.0']
-----
['TEST1', '172.16.0.0', '255.255.0.0']
['TEST1', '172.16.250.5', '255.255.255.255']
-----
['TEST1', '172.16.0.0', '255.255.0.0']
['TEST1', '172.16.250.110', '255.255.255.255']
-----
['TEST1', '172.16.0.0', '255.255.0.0']
['TEST1', '172.16.250.0', '255.255.255.0']
-----
['TEST2', '192.168.1.0', '255.255.254.0']
['TEST2', '192.168.1.0', '255.255.255.0']
-----
[hzw@zeus edo]$
 
 
 
This is the source code, where you can check all the steps. Please test it with your ACLs and let me know about your thoughts.
There are following steps:
  • check if there are duplicates in hosts
  • check if there are duplicates in subnets
  • check if there are overlapping subnets
At the moment I’m aware about following changes which are necessary:
  • improve the output from the 3rd iteration when I check network overlapping, it would be better to not present all combinations (there are duplicates)
  • add an output file as a parameter
  • add a new version  (or parameter) to analyze all encryption domains globally, not only within the same ‘network-group’
 

 

 

 

No comments:

Post a Comment