So back to the original topic, since I have been meaning to write this blog, why the delay? There are two reasons:
1. The documentation and example of the pexpect module is excellent, http://www.noah.org/wiki/pexpect. I wish the examples can be directly posted on the web instead of looking thru the example subfolder, but that is just a small complain. The functions that is mostly used are on the site.
2. I will be a hypocrite if I say I am a complete believer in using this module 100% of the time. At times, such as when hoping thru a Cisco terminal server to access a device thru console, it just became too problematic that rather than debug, I just go with Tcl Explect. It is easy enough to write a small Expect script and use a Python wrapper with subprocess to execute it, much like my post on Expect, http://blog.pythonicneteng.com/2012/09/tcl-expect-with-python-part-1.html.
However, it is still a module that is very useful. Always nice to keep everything in Python so you dont need to go thru the mind-shift of another language.
Here is an example with pexpect that I wrote with the same objective as the Expect script that I posted, http://blog.pythonicneteng.com/2012/09/tcl-expect-with-python-part-1.html. Basically this is what it does:
1. Telnet to AT&T, Comcast, and University of Oregon route server.
2. Execute 'show ip route', and 'show ip bgp ' on the Bing.com IP.
Here is the list of route servers in a text file format:
Here is the list of commands in a text file format:$cat route-serversroute-server.ip.att.netroute-server.newyork.ny.ibone.comcast.netroute-views.oregon-ix.net$
Here is the script itself:$cat commands.txtshow ip route 131.253.13.32show ip bgp 131.253.13.32$
note. these are public route servers, sometimes you will get connection refused if all the vty lines are busy. They are also subject to change with no advance notice. Feel free to try out another route-server from www.traceroute.org.
[update 11/24, took out router = routeServer[0] that was a left over during testing, as a friend correctly pointed out]
$cat routeServ_pexpect.pyHere is the output:
#!/usr/bin/env python
import sys,pexpect
# query file for all route servers and put into a list
routerFile = open('route-servers', 'r')
routeServers = [i for i in routerFile] #look up list comprehension is not clear
# query file for all commands and put into a list
commandFile = open('commands.txt', 'r')
commands = [i for i in commandFile]
# Starts the loop
for router in routeServers:
child = pexpect.spawn ('telnet', [router.strip()]) #option needs to be a list
child.logfile = sys.stdout #display progress on screen
child.expect ('Username:')
child.sendline('rviews')
for command in commands:
child.expect(['route-server', 'route-views']) #different options on prompt
child.sendline(command)
child.expect(['route-server', 'route-views'])
$
$./routeServ_pexpect.pyHappy Holidays!
Trying 12.0.1.28...
Connected to route-server.cbbtier3.att.net.
Escape character is '^]'.
-------------- route-server.ip.att.net ---------------
--------- AT&T IP Services Route Monitor -----------
The information available through route-server.ip.att.net is offered
by AT&T's Internet engineering organization to the Internet community.
This router maintains eBGP peerings with customer-facing routers
throughout the AT&T IP Services Backbone:
IPv4:
12.123.21.243 Atlanta 12.123.133.124 Austin 12.123.41.250 Cambridge
12.123.5.240 Chicago 12.123.17.244 Dallas 12.123.139.124 Detroit
12.123.37.250 Denver 12.123.134.124 Houston 12.123.29.249 LA
12.123.1.236 New York 12.123.33.249 Orlando 12.123.137.124 Philly
12.123.142.124 Phoenix 12.123.145.124 SanDiego 12.123.13.241 SanFran
12.123.25.245 St.Louis 12.123.45.252 Seattle 12.123.9.241 WashDC
IPv6:
2001:1890:FF:FFFF:12:122:124:12 Atlanta
2001:1890:FF:FFFF:12:122:127:66 Chicago
2001:1890:FF:FFFF:12:122:124:138 Dallas
2001:1890:FF:FFFF:12:122:120:7 Fort Lauderdale
2001:1890:FF:FFFF:12:122:125:6 Los Angeles
2001:1890:FF:FFFF:12:122:125:44 New York
2001:1890:FF:FFFF:12:122:125:106 Philadelphia
2001:1890:FF:FFFF:12:122:125:132 Phoenix
2001:1890:FF:FFFF:12:122:126:232 San Francisco
2001:1890:FF:FFFF:12:122:125:224 Seattle
2001:1890:FF:FFFF:12:122:126:9 St. Louis
2001:1890:FF:FFFF:12:122:126:64 Washington
*** Please Note:
Ping and traceroute delay figures measured here are unreliable, due to the
high CPU load experienced when complicated "show" commands are running.
For questions about this route-server, send email to: jayb@att.com
*** Log in with username "rviews", no password required ***
User Access Verification
Username: Kerberos: No default realm defined for Kerberos!
rviews
rviews
route-server>show ip route 131.253.13.32
show ip route 131.253.13.32
Routing entry for 131.253.12.0/22
Known via "bgp 65000", distance 20, metric 0
Tag 7018, type external
Last update from 12.123.1.236 4w1d ago
Routing Descriptor Blocks:
* 12.123.1.236, from 12.123.1.236, 4w1d ago
Route metric is 0, traffic share count is 1
AS Hops 3
Route tag 7018
route-server>
route-server>show ip bgp 131.253.13.32
show ip bgp 131.253.13.32
BGP routing table entry for 131.253.12.0/22, version 711761
Paths: (18 available, best #13, table Default-IP-Routing-Table)
Not advertised to any peer
7018 8075 8075, (received & used)
12.123.133.124 from 12.123.133.124 (12.123.133.124)
Origin IGP, localpref 100, valid, external
Community: 7018:2500 7018:36244
7018 8075 8075, (received & used)
12.123.21.243 from 12.123.21.243 (12.123.21.243)
Origin IGP, localpref 100, valid, external
Community: 7018:2500 7018:36244
7018 8075 8075, (received & used)
12.123.139.124 from 12.123.139.124 (12.123.139.124)
Origin IGP, localpref 100, valid, external
Community: 7018:2500 7018:34011
7018 8075 8075, (received & used)
12.123.33.249 from 12.123.33.249 (12.123.33.249)
Origin IGP, localpref 100, valid, external
Community: 7018:2500 7018:36244
7018 8075 8075, (received & used)
12.123.17.244 from 12.123.17.244 (12.123.17.244)
Origin IGP, localpref 100, valid, external
Community: 7018:2500 7018:36244
route-server>Trying 66.208.229.1...
Connected to route-server.newyork.ny.ibone.comcast.net.
Escape character is '^]'.
********************************************************************************
Comcast NETO Route Server
This route server is provided by Comcast National Engineering &
Technical Operations to provide visibility into the Internet routing table
from the perspective of Comcast's network. This server has BGP peerings to
the following routers and the Internet routing table view from each device:
68.86.80.0 - Ashburn, VA
68.86.80.2 - New York, NY
68.86.80.5 - Chicago, IL
68.86.80.6 - Chicago, IL 2
68.86.80.7 - Denver, CO
68.86.80.10 - San Jose, CA
68.86.80.11 - Los Angeles, CA
68.86.80.12 - Atlanta, GA
68.86.80.13 - Dallas, TX
Note: Due to high CPU utilization on this device, ping and traceroute results
may be unreliable. This route server should not be used to measure network
performance as a result.
Login with username: rviews
Location: New York City
Network: Comcast Route Server
********************************************************************************
User Access Verification
Username: rviews
Kerberos: No default realm defined for Kerberos!
rviews
route-server.newyork.ny.ibone>show ip route 131.253.13.32
show ip route 131.253.13.32
Command authorization failed.
route-server.newyork.ny.ibone>
route-server.newyork.ny.ibone>show ip bgp 131.253.13.32
show ip bgp 131.253.13.32
BGP routing table entry for 131.253.12.0/22, version 1517562788
Paths: (8 available, best #8, table Default-IP-Routing-Table)
Advertised to update-groups:
2
8075 8075, (received & used)
68.86.1.38 (metric 81950) from 68.86.80.10 (68.86.1.10)
Origin IGP, metric 0, localpref 300, valid, internal
Community: 7922:38 7922:3020
Originator: 68.86.1.38, Cluster list: 68.86.1.10
8075 8075, (received & used)
68.86.1.38 (metric 81950) from 68.86.80.11 (68.86.1.11)
Origin IGP, metric 0, localpref 300, valid, internal
Community: 7922:38 7922:3020
Originator: 68.86.1.38, Cluster list: 68.86.1.11, 68.86.1.10
8075 8075, (received & used)
68.86.1.41 (metric 73815) from 68.86.80.13 (68.86.1.13)
Origin IGP, metric 0, localpref 300, valid, internal
Community: 7922:41 7922:3020
Originator: 68.86.1.41, Cluster list: 68.86.1.13
8075 8075, (received & used)
68.86.1.40 (metric 69750) from 68.86.80.12 (68.86.1.12)
Origin IGP, metric 0, localpref 300, valid, internal
Community: 7922:40 7922:3020
route-server.newyork.ny.ibone>Trying 128.223.51.103...
Connected to route-views.oregon-ix.net.
Escape character is '^]'.
**********************************************************************
Oregon Exchange BGP Route Viewer
route-views.oregon-ix.net / route-views.routeviews.org
route views data is archived on http://archive.routeviews.org
This hardware is part of a grant from Cisco Systems.
Please contact help@routeviews.org if you have questions or
comments about this service, its use, or if you might be able to
contribute your view.
This router has views of the full routing tables from several ASes.
The list of ASes is documented under "Current Participants" on
http://www.routeviews.org/.
**************
route-views.routeviews.org is now using AAA for logins. Login with
username "rviews". See http://routeviews.org/aaa.html
**********************************************************************
User Access Verification
Username: rviews
rviews
route-views>show ip route 131.253.13.32
show ip route 131.253.13.32
Routing entry for 131.253.12.0/22
Known via "bgp 6447", distance 20, metric 100
Tag 8075, type external
Last update from 207.46.32.34 3d07h ago
Routing Descriptor Blocks:
* 207.46.32.34, from 207.46.32.34, 3d07h ago
Route metric is 100, traffic share count is 1
AS Hops 2
Route tag 8075
route-views>
route-views>show ip bgp 131.253.13.32
show ip bgp 131.253.13.32
BGP routing table entry for 131.253.12.0/22, version 218212
Paths: (33 available, best #7, table Default-IP-Routing-Table)
Not advertised to any peer
852 8075 8075
154.11.11.113 from 154.11.11.113 (154.11.11.113)
Origin IGP, metric 0, localpref 100, valid, external
Community: 852:180
3356 8075 8075
4.69.184.193 from 4.69.184.193 (4.69.184.193)
Origin IGP, metric 0, localpref 100, valid, external
Community: 209:80 209:888 701:30 703:30 1120:2 1299:2869 1299:5509 1299:5583 2914:470 3356:3 3356:22 3356:100 3356:123 3356:575 3356:2012 3549:200 3549:666 4657:905 5511:1550 6453:80 6762:20008 7018:80 7473:22009 7473:23009 10026:11010 10026:11020 10026:11030 10026:11040 10026:11050 10026:11060 10026:11070 10026:11080 10026:11090 10026:11100 25160:209 25160:1239 65009:7018
4436 8075 8075
69.31.111.244 from 69.31.111.244 (69.31.111.244)
Origin IGP, metric 0, localpref 100, valid, external
Community: 209:80 209:888 701:30 703:30 1120:2 1299:2869 1299:5509 1299:5583 2914:470 3549:200 3549:666 4436:21216 4657:905 5511:1550 6453:80 6762:20008 7018:80 7473:22009 7473:23009 10026:11010 10026:11020 10026:11030 10026:11040 10026:11050 10026:11060 10026:11070 10026:11080 10026:11090 10026:11100 25160:209 25160:1239 65009:7018
route-views>$
$
Eric,
ReplyDeleteGreat post, but I think the real interesting question is the one you hinted at, but did not actually ask. I agree that it is depressing that we still screen scrape after all these years. I am amazed at how many people do not even know how to code basic screen scraping though. It seems to me, we need a paradigm shift, and a few tools to enable that shift.
We need router vendors to support some basic functions in the O/S in support of automation. Imagine a world where you could open an ssh command channel, pass an object, ask the vendor to confirm the object was syntactically valid, and ask for a provisional commit with 60 second rollback if keep-alive was lost, and some options for timed (10 minute, 10 hour or whatever) rollback pending a smoke-test. If the if-then regex based logic of your screen scraping was handled by the OS and you just passed commands and examined return codes from atomic operations, life would be easier.
In that world, you would want to require a reasonable smoke test functions as a core part of the service definition for any service offerings pre-deployment. If you cannot define a good smoke test, then the service is not deployed. And the servers and services need API (interfaces) to support smoke testing. We do things like http GET test-object.html for testing, instead of having html primitives to support smoke testing. We telnet 25 to mail servers and expn or vrfy pretend users, instead of having programmatic interfaces to mail servers. For every common server there is a reasonable set of functions that it ought to support. We have some SNMP MIBs for this sort of thing, and you can load SNMP modules into python and expect, but it seems like the interfaces are not as clean and as unified as might be desirable. It seems like the service is defined and the monitoring is shimmed in after the service definition, instead of making it integral to the service offering. I realize the Internet is a moving target, and trying to define this framework would be like shooting at missile with a slingshot while strapped to a race car that was being upgraded as it was being driven, but I think it is still an idea worth thinking about.
But that only addresses a rational framework for pushing, pulling and changing configurations on networks using today's paradigms. An old Colleague Marek J. asked the question, "Why do we build networks from discrete components, instead of building a service around agglutinated sets of devices, configured from a single service interface." He used fewer and smaller words, but that was the gist of the conversation. Software Defined Networking is doing some of that, but it is not clear to me if vendors are really shooting for adding value to the process, or simply trying to find ways to ensure they are not removed from the value chain as hardware vendors make servers, switches and routers into commodities, and SDN attempts to provide open source control of those commodity components.
Elle Plato
Thanks Elle, points well taken. :)
DeleteTruth be told, fast Web data transmission won't be modest. Running your system whilst not securing it much can likewise be amazingly dangerous. There are steps you can without much of a stretch take to ensure that your wi-fi home system will be protected. ws c3750x 24s s
ReplyDeleteThanx for your code )
ReplyDeleteWhy my cycle execute only once ? can't understand
#!/usr/bin/env python
import sys,pexpect
# query file for all route servers and put into a list
routerFile = open('IP_list.txt', 'r')
routeServers = [i for i in routerFile] #look up list comprehension is not clear
# query file for all commands and put into a list
commandFile = open('command.txt', 'r')
commands = [i for i in commandFile]
# Starts the loop
for router in routeServers:
child = pexpect.spawn ('telnet', [router]) #option needs to be a list
child.logfile = sys.stdout #display progress on screen
child.expect (':')
child.sendline ('user')
child.expect (':')
child.sendline('password')
for command in commands:
child.expect('#') #different options on prompt
child.sendline(command)
child.expect('#')
Not sure if it was copy / paste formatting, but the loop indentation was lost in the post. Maybe that was it?
DeleteNo, my script has correct indention, that input form kill them )
DeleteI have one more question, how can i increase searchwindowsize and maxread ? i use that line :
child = pexpect.spawn ('telnet', [router.strip()],maxread=1000000, searchwindowsize=20000)
but tty output still not change
Nice blog
ReplyDeleteangular4 interview questions
python interview questions
artificial intelligence interview questions
python online training
artificial intelligence online training
I got nice blog
ReplyDeletesap partner companies in bangalore
sap implementation companies in bangalore
sap partners in india
aws staffing
jquery interview questions
sql interview questions
Thank you for sharing this type of interview questions
ReplyDeleteIot Online Training
Itil Interview Questions
Salesforce Interview Questions
Msbi Interview questions
Salesforce Interview Questions
C Interview Questions
aws partners in bangalore
aws partners in india
aws service providers in india
aws service providers
Thank you for Posting
ReplyDeletesap partner companies in bangalore
sap implementation companies in bangalore
sap partners in india
aws staffing
Hi There,
ReplyDeleteI love all the posts, I really enjoyed.
I would like more information about this, because it is very nice., Thanks for sharing.
i wrote a function two different ways. can someone tell me which is better and why?
Python Code: (Double-click to select all)
1
2
3
4
5
6
7
8
9 def pathftn(p,unknown='?'):
if p.is_symlink: return 'l'
if p.is_block_device: return 'b'
if p.is_char_device: return 'c'
if p.is_dir: return 'd'
if p.is_file: return 'f'
if p.is_socket: return 's'
if p.is_fifo: return 'p'
return unknown
Python Code: (Double-click to select all)
1
2
3
4
5
6
7
8
9
10 def pathftn(p,unknown='?'):
t=unknown
if p.is_symlink: t='l'
elif p.is_block_device: t='b'
elif p.is_char_device: t='c'
elif p.is_dir: t='d'
elif p.is_file: t='f'
elif p.is_socket: t='s'
elif p.is_fifo: t='p'
return t
Excellent tutorials - very easy to understand with all the details. I hope you will continue to provide more such tutorials.
Thanks,
Irene Hynes
I have to voice my passion for your kindness giving support to those people that should have guidance on this important matter.
ReplyDeleteAWS Training in Bangalore
i tried but its taking only first line of the "commands.txt".
ReplyDeletecommands.txt contains:
sh run vlan 158
sh run vlan 159
def shvlan(device,child):
commandFile = open('commands.txt', 'r')
commands = [i for i in commandFile]
for command in commands:
child.sendline(command)
child.expect('.*#')
vlans = child.after
child.close()
print device + ' conn closed'
print 'sh run vlan executed'
print vlans
return vlans
it is very important
ReplyDeletesluts mature
hi Eric,
ReplyDeletethe same output if incase i want to store into another text file how can we do that? can you post the extemnsion of this code for same?
Wonderful Blog. Keep Posting.
ReplyDeleteAdvanced Excel Training in Chennai
Corporate Excel Training in Mumbai
Advanced Excel Training in Bangalore
Power BI Training in Chennai
Corporate Tableau Training
Corproate Excel Training Delhi, Gurgaon, Noida
Thank you for sharing your valuable suggesions its very usefull to me Python Training, SVR Technologies offers Online & Corporate Classes with Free Live Demo, We cover practical sessions and all the modules in Python..
ReplyDelete24/7 Online Training | Corporate Training, Videos, Job Support. SVR Technologies building careers for Software Engineers With online Training self-paced.
Features:
>> Live Instructor LED Classes
>> Experienced Faculty
>> Free Video materials
>> 24/7 Support
>> Flexible Timings
>> Lowest Fee
Enroll For Live Classes: python online training
The 3D laser scanning service is very popular nowadays. Because 3D laser scanning is very useful for project surveying, design, and construction. If you need the best 3D laser scanning service then you can visit our site. Best 3D laser scanning service Calgary, Alberta
ReplyDeleteThis blog is very creative and very interesting. After a long time, I see the great blog with a good explanation about this topic and this post is very useful for me. I like to you more unique post, please update them...
ReplyDeleteOracle Training in Chennai
Oracle course in Chennai
Tableau Training in Chennai
Spark Training in Chennai
Excel Training in Chennai
Primavera Training in Chennai
Appium Training in Chennai
Power BI Training in Chennai
Oracle Training in T Nagar
Oracle Training in Porur