*****
[Update 11/25/2016: I humbly invite you to check out my InternetworkExpert Class on Learn SDN with OpenFlow and Ryu Controller]
*****
The OpenFlow Tutorial is simply awesome. It is hands down the best way to gain some hands on experience with OpenFlow. Everything is encapsulated into a VirtualBox VM and you simply need to bootstrap it. Everything you need for network component is included in Mininet, which is another awesome tool in itself. The wireshark component has the OpenFlow dissect included already. And to run simple OpenFlow test you can use the built-in dpctl.
I do, however, think when it comes to actually construct a Controller + Mininet network the tutorial needs more example and hand-holding. Let's face it, for those of us coming from Cisco / Juniper background with some Python chops it is still a very steep learning curve when:
[If this is helpful to you, please leave me a note. If enough people respond, we can add a few items to the tutorial wiki. Otherwise I will just keep it here on my blog]
1. You dont know which portion is OpenFlow, which is Python, and which is Controller.
For example, when the tutorial reference of.ofp_flow_mod() and "this instructs a switch to install a flow table entry"; it includes all three components, POX/OpenFlow/Python. The flow_mod is just an OpenFlow message, while it is implemented by POX via a Python function. But it is not until I read the POX WIki that I realize the the function is implemented in the "pox.openflow.libopenflow_01" module and the example probably skipped the potion where 'import pox.openflow.libopenflow_01 as of' at the top of the file.
2. Most of the pointers later on are task oriented, there is no end-to-end examples to build from.
So here I aim to pick up at the "Create Learning Switch" section and provide a more detailed example using POX as the controller.
[Update 04/06/2013: If you are using the new VM with mininet 2.0 (added Feb 2013) in the tutorial, the new s1 switch port 1 and port 2 now connects to h1 and h2 instead of h2 and h3. The section below used the old image, which uses h2 and h3 at the first two ports. Please adjust accordingly if you use the new image.
old:
Old:
openflow@openflowtutorial:~$ sudo mn --topo single,3 --mac --switch ovsk --controller remote
*** Adding links:
(s1, h2) (s1, h3) (s1, h4)
***
mininet> net
s1 <-> h2-eth0 h3-eth0 h4-eth0
mininet>
New:
mininet@mininet-vm:~$ sudo mn --topo single,3 --mac --switch ovsk --controller remote
*** Adding links:
(h1, s1) (h2, s1) (h3, s1)
***
mininet> net
c0
s1 lo: s1-eth1:h1-eth0 s1-eth2:h2-eth0 s1-eth3:h3-eth0
h1 h1-eth0:s1-eth1
h2 h2-eth0:s1-eth2
h3 h3-eth0:s1-eth3
mininet>
]
1. Clone the POX controller from Git repository, Git is already pre-installed on the VM:
openflow@openflowtutorial:~$ git clone http://github.com/noxrepo/pox
Cloning into pox…
2. Verify and change into the directory:
openflow@openflowtutorial:~$ ls
mininet nox oflops oftest openflow openvswitch pox
openflow@openflowtutorial:~$ cd pox
openflow@openflowtutorial:~/pox$ ls
COPYING debug-pox.py doc ext pox pox.py README setup.cfg tests tools
openflow@openflowtutorial:~/pox$
openflow@openflowtutorial:~$ sudo mn --topo single,3 --mac --switch ovsk --controller remote4. Test your POX controller by starting the POX, look for the switch registration message. You can optionally open up wireshark to verify:
*** Adding controller
*** Creating network
*** Adding hosts:
h2 h3 h4
*** Adding switches:
s1
*** Adding links:
(s1, h2) (s1, h3) (s1, h4)
*** Configuring hosts
h2 h3 h4
*** Starting controller
*** Starting 1 switches
s1
*** Starting CLI:
mininet>
openflow@openflowtutorial:~/pox$ ./pox.py
POX 0.0.0 / Copyright 2011 James McCauley
DEBUG:core:POX 0.0.0 going up...
DEBUG:core:Running on CPython (2.7.1+/Apr 11 2011 18:05:24)
INFO:core:POX 0.0.0 is up.
This program comes with ABSOLUTELY NO WARRANTY. This program is free software,
and you are welcome to redistribute it under certain conditions.
Type 'help(pox.license)' for details.
DEBUG:openflow.of_01:Listening for connections on 0.0.0.0:6633
INFO:openflow.of_01:[Con 1/1] Connected to 00-00-00-00-00-01
Ready.
POX>
POX>
5. If you go back to the Mininet window and try to have H2 ping H3, nothing will happen because the controller is not doing anything. This is pretty boring.
mininet> h2 ping h3
<nothing>
<ctrl+c to kill>
6. At this point, exit the interactive prompt with exit(). You can leave the Mininet topology running or kill it in another window. I find it easier to kill it and just 'up arrow + enter' every time to see my controller messages.
POX> exit()7. The way POX initiate components, is to use the component name as the argument. To use a pre-built L2 learning switch component, you can use 'forwarding.l2_learning' as the first argument.
openflow@openflowtutorial:~/pox$ ./pox.py forwarding.l2_learning
POX 0.0.0 / Copyright 2011 James McCauley
DEBUG:core:POX 0.0.0 going up...
DEBUG:core:Running on CPython (2.7.1+/Apr 11 2011 18:05:24)
INFO:core:POX 0.0.0 is up.
This program comes with ABSOLUTELY NO WARRANTY. This program is free software,
and you are welcome to redistribute it under certain conditions.
Type 'help(pox.license)' for details.
DEBUG:openflow.of_01:Listening for connections on 0.0.0.0:6633
Ready.
POX>
POX> INFO:openflow.of_01:[Con 1/1] Connected to 00-00-00-00-00-01
DEBUG:forwarding.l2_learning:Connection [Con 1/1]
7. At this point, if you go back to Mininet window, you can see that as a learning switch, h2 can now poing h3:
8. So now we know POX works, let's start with some boiler plate code so we can learn. Toward the end of the POX Wiki, there is a link for the code prepared by William Emmanuel Yu (not sure if all the codes were written by him). Get his codes from Git as well.
11. For example, to build a dumb hub, follow the code and only write out what is required. Here is a block-by-block:
mininet> h2 ping -c5 h3And your POX controller prompt should show the controller installing the flows:
PING 10.0.0.3 (10.0.0.3) 56(84) bytes of data.
64 bytes from 10.0.0.3: icmp_req=1 ttl=64 time=17.4 ms
64 bytes from 10.0.0.3: icmp_req=2 ttl=64 time=0.298 ms
64 bytes from 10.0.0.3: icmp_req=3 ttl=64 time=0.041 ms
64 bytes from 10.0.0.3: icmp_req=4 ttl=64 time=0.050 ms
64 bytes from 10.0.0.3: icmp_req=5 ttl=64 time=0.000 ms
--- 10.0.0.3 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4007ms
rtt min/avg/max/mdev = 0.000/3.559/17.407/6.924 ms
mininet>
POX>You can also use the dpctl tool to verify that the flows are installed:
POX> DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:03.2 -> 00:00:00:00:00:02.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:02.1 -> 00:00:00:00:00:03.2
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:03.2 -> 00:00:00:00:00:02.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:03.2 -> 00:00:00:00:00:02.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:02.1 -> 00:00:00:00:00:03.2
POX>
openflow@openflowtutorial:~/pox$ dpctl dump-flows tcp:127.0.0.1:6634Notice the first ICMP reply is really slow, waiting for the flow to be installed. But later packets are fast.
stats_reply (xid=0xad1245f1): flags=none type=1(flow)
cookie=0, duration_sec=4s, duration_nsec=993000000s, table_id=0, priority=32768, n_packets=1, n_bytes=42, idle_timeout=10,hard_timeout=30,arp,dl_vlan=0xffff,dl_vlan_pcp=0x00,dl_src=00:00:00:00:00:02,dl_dst=00:00:00:00:00:03,nw_src=10.0.0.2,nw_dst=10.0.0.3,nw_tos=0x00,nw_proto=2,tp_src=0,tp_dst=0,actions=output:2
cookie=0, duration_sec=9s, duration_nsec=1000000s, table_id=0, priority=32768, n_packets=7, n_bytes=686, idle_timeout=10,hard_timeout=30,icmp,dl_vlan=0xffff,dl_vlan_pcp=0x00,dl_src=00:00:00:00:00:02,dl_dst=00:00:00:00:00:03,nw_src=10.0.0.2,nw_dst=10.0.0.3,nw_tos=0x00,icmp_type=8,icmp_code=0,actions=output:2
cookie=0, duration_sec=4s, duration_nsec=994000000s, table_id=0, priority=32768, n_packets=1, n_bytes=42, idle_timeout=10,hard_timeout=30,arp,dl_vlan=0xffff,dl_vlan_pcp=0x00,dl_src=00:00:00:00:00:03,dl_dst=00:00:00:00:00:02,nw_src=10.0.0.3,nw_dst=10.0.0.2,nw_tos=0x00,nw_proto=1,icmp_type=0,icmp_code=0,actions=output:1
cookie=0, duration_sec=10s, duration_nsec=32000000s, table_id=0, priority=32768, n_packets=9, n_bytes=882, idle_timeout=10,hard_timeout=30,icmp,dl_vlan=0xffff,dl_vlan_pcp=0x00,dl_src=00:00:00:00:00:03,dl_dst=00:00:00:00:00:02,nw_src=10.0.0.3,nw_dst=10.0.0.2,nw_tos=0x00,icmp_type=0,icmp_code=0,actions=output:1
openflow@openflowtutorial:~/pox$
8. So now we know POX works, let's start with some boiler plate code so we can learn. Toward the end of the POX Wiki, there is a link for the code prepared by William Emmanuel Yu (not sure if all the codes were written by him). Get his codes from Git as well.
openflow@openflowtutorial:~/pox$ git clone https://github.com/hip2b2/poxstuff.gitNote. There is also the interactive version of the same of_sw_tutorial_oo.py that is listed in the POX Wiki. But it did not work for me. Gave me the error below. I also prefer to do the 'dumb' thing and just type out code to help me learn. Just FYI.
Cloning into poxstuff...
remote: Counting objects: 70, done.
remote: Compressing objects: 100% (41/41), done.
remote: Total 70 (delta 39), reused 59 (delta 28)
Unpacking objects: 100% (70/70), done.
openflow@openflowtutorial:~/pox$
openflow@openflowtutorial:~/pox$ ./pox.py of_sw_tutorial_oo9. POX controller will automatically look for the ext/ directory for components. You can copy the of_switch_tutorial.py file to the ext/ folder if you want the POX controller to be able to find it automatically.
POX 0.0.0 / Copyright 2011 James McCauley
DEBUG:ext.of_sw_tutorial_oo:Initializing switch SW_IDEALPAIRSWITCH.
Traceback (most recent call last):
File "./pox.py", line 352, in main
if doLaunch():
File "./pox.py", line 152, in doLaunch
f(**params)
File "/home/openflow/pox/ext/of_sw_tutorial_oo.py", line 327, in launch
core.Interactive.variables['MySwitch'] = MySwitch
File "/home/openflow/pox/pox/core.py", line 346, in __getattr__
raise AttributeError("'%s' not registered" % (name,))
AttributeError: 'Interactive' not registeredopenflow@openflowtutorial:~/pox$
openflow@openflowtutorial:~/pox$ cp poxstuff/of_sw_tutorial.py ext/10. To build muscle memory, I manually create a Python file, can start typing in line-by-line in order to learn. Here is the link I look at, https://github.com/hip2b2/poxstuff/blob/master/of_sw_tutorial.py.
openflow@openflowtutorial:~/pox$ ./pox.py of_sw_tutorial
POX 0.0.0 / Copyright 2011 James McCauley
INFO:ext.of_sw_tutorial:Switch Tutorial is running.
DEBUG:core:POX 0.0.0 going up...
DEBUG:core:Running on CPython (2.7.1+/Apr 11 2011 18:05:24)
INFO:core:POX 0.0.0 is up.
This program comes with ABSOLUTELY NO WARRANTY. This program is free software,
and you are welcome to redistribute it under certain conditions.
Type 'help(pox.license)' for details.
DEBUG:openflow.of_01:Listening for connections on 0.0.0.0:6633
Ready.
POX>
POX> INFO:openflow.of_01:[Con 1/1] Connected to 00-00-00-00-00-01
11. For example, to build a dumb hub, follow the code and only write out what is required. Here is a block-by-block:
1 #!/usr/bin/env python- The pox.core and pox.openflow.libopenflow_01 are the packages being in use in the functions:
2
3 # Copy and paste https://github.com/hip2b2/poxstuff/blob/master/of_sw_tutorial.py
4 # step by step to understand the process
5
6 from pox.core import core- Logging for output:
7 import pox.openflow.libopenflow_01 as of
8
9 log = core.getLogger()- This is a base function to allow sending packets out:
10
11 table = {}
12
13 def send_packet(event, dst_port = of.OFPP_ALL):- Here is the dumb hub function itself:
14 msg = of.ofp_packet_out(in_port=event.ofp.in_port)
15 if event.ofp.buffer_id != -1 and event.ofp.buffer_id is not None:
16 msg.buffer_id = event.ofp.buffer_id
17 else:
18 if event.ofp.data:
19 return
20 msg.data = event.ofp.data
21 msg.actions.append(of.ofp_action_output(port = dst_port))
22 event.connection.send(msg)
23
24 def _handle_dumbhub_packetin(event):- Here is the launch() function to specify which function to use:
25 packet = event.parsed
26 send_packet(event, of.OFPP_ALL)
27
28 log.debug("Broadcasting %s.%i -> %s.%i" %
29 (packet.src, event.ofp.in_port, packet.dst, of.OFPP_ALL))
30
31 # launch whichever implementation you want via function
32 def launch():
33 core.openflow.addListenerByName("PacketIn", _handle_dumbhub_packetin)
34
35 log.info("Switch Tutorial is running.")
36
12. Let's start our simple dumb hub controller:
openflow@openflowtutorial:~/pox$ ./pox.py of_sw_tutorial_myTestPOX 0.0.0 / Copyright 2011 James McCauley
INFO:ext.of_sw_tutorial_myTest:Switch Tutorial is running.
DEBUG:core:POX 0.0.0 going up...
DEBUG:core:Running on CPython (2.7.1+/Apr 11 2011 18:05:24)
INFO:core:POX 0.0.0 is up.
This program comes with ABSOLUTELY NO WARRANTY. This program is free software,
and you are welcome to redistribute it under certain conditions.
Type 'help(pox.license)' for details.
DEBUG:openflow.of_01:Listening for connections on 0.0.0.0:6633
Ready.
POX>
POX> INFO:openflow.of_01:[Con 1/1] Connected to 00-00-00-00-00-01
mininet> h2 ping -c5 h3
PING 10.0.0.3 (10.0.0.3) 56(84) bytes of data.
64 bytes from 10.0.0.3: icmp_req=1 ttl=64 time=74.3 ms
64 bytes from 10.0.0.3: icmp_req=2 ttl=64 time=33.4 ms
64 bytes from 10.0.0.3: icmp_req=3 ttl=64 time=44.9 ms
64 bytes from 10.0.0.3: icmp_req=4 ttl=64 time=21.6 ms
64 bytes from 10.0.0.3: icmp_req=5 ttl=64 time=6.10 ms
--- 10.0.0.3 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4026ms
rtt min/avg/max/mdev = 6.101/36.093/74.358/23.053 ms
mininet>
Notice the packet time are longer on the first packet, and subsequent packets have longer time compare to the L2 learning switch.
You will also see the broadcast packets on the POX console:
POX> DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:02.1 -> ff:ff:ff:ff:ff:ff.65532
DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:03.2 -> 00:00:00:00:00:02.65532
DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:02.1 -> 00:00:00:00:00:03.65532
DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:03.2 -> 00:00:00:00:00:02.65532
DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:02.1 -> 00:00:00:00:00:03.65532
DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:03.2 -> 00:00:00:00:00:02.65532
DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:02.1 -> 00:00:00:00:00:03.65532
DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:03.2 -> 00:00:00:00:00:02.65532
DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:02.1 -> 00:00:00:00:00:03.65532
DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:03.2 -> 00:00:00:00:00:02.65532
DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:02.1 -> 00:00:00:00:00:03.65532
DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:03.2 -> 00:00:00:00:00:02.65532
DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:03.2 -> 00:00:00:00:00:02.65532
DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:02.1 -> 00:00:00:00:00:03.65532
13. You can now start to experiment with the functions. Say, you can insert a print statement to see what the parsed packet look like:
23
24 def _handle_dumbhub_packetin(event):
25 packet = event.parsed
26 print packet 27 send_packet(event, of.OFPP_ALL)
28
29 log.debug("Broadcasting %s.%i -> %s.%i" %
30 (packet.src, event.ofp.in_port, packet.dst, of.OFPP_ALL))
31
Restart the controller, you will now see the packet:
POX> INFO:openflow.of_01:[Con 1/1] Connected to 00-00-00-00-00-01
[00:00:00:00:00:02>00:00:00:00:00:03:IP]|([v:4hl:5l:84t:64]ICMP cs:26a5[10.0.0.2>10.0.0.3]){t:ECHO_REQUEST c:0 chk:23e7}{id:2835 seq:3}DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:02.1 -> 00:00:00:00:00:03.65532
[00:00:00:00:00:03>00:00:00:00:00:02:IP]|([v:4hl:5l:84t:64]ICMP cs:dfc5[10.0.0.3>10.0.0.2]){t:ECHO_REPLY c:0 chk:2be7}{id:2835 seq:3}DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:03.2 -> 00:00:00:00:00:02.65532
[00:00:00:00:00:02>00:00:00:00:00:03:IP]|([v:4hl:5l:84t:64]ICMP cs:26a5[10.0.0.2>10.0.0.3]){t:ECHO_REQUEST c:0 chk:4fd1}{id:2835 seq:4}
14. Or if you want to look at the packet out message:
13 def send_packet(event, dst_port = of.OFPP_ALL):
14 msg = of.ofp_packet_out(in_port=event.ofp.in_port)
15 print msg 16 if event.ofp.buffer_id != -1 and event.ofp.buffer_id is not None:
17 msg.buffer_id = event.ofp.buffer_id
18 else:
19 if event.ofp.data:
20 return
21 msg.data = event.ofp.data
22 msg.actions.append(of.ofp_action_output(port = dst_port))
23 event.connection.send(msg)
24
POX> ofp_packet_out header: version: 1 type: 13 (OFPT_PACKET_OUT) length: 8 xid: None buffer_id: -1 in_port: 1 actions_len: 0 actions:DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:02.1 -> 00:00:00:00:00:03.65532
ofp_packet_out header: version: 1 type: 13 (OFPT_PACKET_OUT) length: 8 xid: None buffer_id: -1 in_port: 2 actions_len: 0 actions:DEBUG:ext.of_sw_tutorial_myTest:Broadcasting 00:00:00:00:00:03.2 -> 00:00:00:00:00:02.65532
15. Let's try the lazyhub function, just create the lazyhub function and change the launch() function:
23
24 def _handle_lazyhub_packetin (event): 25 packet = event.parsed 26 27 msg = of.ofp_flow_mod() 28 msg.idle_timeout = 10 29 msg.hard_timeout = 30 30 msg.actions.append(of.ofp_action_output(port = of.OFPP_ALL)) 31 event.connection.send(msg) 32 33 log.debug("Installing %s.%i -> %s.%i" % 34 ("ff:ff:ff:ff:ff:ff", event.ofp.in_port, "ff:ff:ff:ff:ff:ff", of.OFPF_ALL)) 35 36
37 # launch whichever implementation you want via function
38 def launch():
39 core.openflow.addListenerByName("PacketIn", _handle_lazyhub_packetin)
40
41 log.info("Switch Tutorial is running.")
42
mininet> h2 ping -c5 h3
PING 10.0.0.3 (10.0.0.3) 56(84) bytes of data.
64 bytes from 10.0.0.3: icmp_req=2 ttl=64 time=0.674 ms
64 bytes from 10.0.0.3: icmp_req=3 ttl=64 time=0.060 ms
64 bytes from 10.0.0.3: icmp_req=4 ttl=64 time=0.041 ms
64 bytes from 10.0.0.3: icmp_req=5 ttl=64 time=0.286 ms
--- 10.0.0.3 ping statistics ---
5 packets transmitted, 4 received, 20% packet loss, time 4014ms
rtt min/avg/max/mdev = 0.041/0.265/0.674/0.255 ms
mininet>
POX>
POX> INFO:openflow.of_01:[Con 1/1] Connected to 00-00-00-00-00-01
DEBUG:ext.of_sw_tutorial_myTest:Installing ff:ff:ff:ff:ff:ff.1 -> ff:ff:ff:ff:ff:ff.65532
POX>
I am not sure if this is helpful for others, but at least for me, this provides me with a good entry point to start understanding the different aspects of the POX controller and to proceed.
Cheers. Leave me comments to let me know how to improve.
HI Eric Chou, i have some question, hope you'll help me:
ReplyDelete-when i created topo like you first: & sudo mn --topo single,3 --mac --switch ovsk --controller remote , i received message :Unable to contact the remote controller at 127.0.0.1:6633
-when i tried connect to POX: ~/pox$ ./pox.py, i only saw display: INFO:openflow.of_01:[Con 1/1] Connected to 00-00-00-00-00-01 and no more. I saw you have POX> in below, i think this is place to make code and can controlle in POX.
This is 2 my question. I'm the beginner at mininet.my email: tuanhai.bk@gmail.com
Hi Hai Pham, thanks for reading! I took down my VM at the time to save some laptop resources. But both of those messages you saw are normal. The mininet topology is created with 1 switch and 3 hosts, the switch is trying to connect to the kernel switch at port 6633. If you havn't started the POX controller at port 6633, then it would not connect.
DeleteWhen the POX controller is started, the switch immediately connects to the controller. The mininet sets the switch mac address as 00:00:00:00:00:01 (hosts are 0::02, 0::03, 0:004, etc IIRC). But by default, there is no applications associated with the POX, so you should proceed to step 6, stop the POX controller after you make sure the switch can register. Then proceed to use the L2 Learning switch app to make sure everything is working.
If you are happy with the boiler plate applications (dumb hub, L2 learning switch), then you can stop there. But if you are interested in learning how to make your own application, then you should proceed.
Hope it helps. :)
Hey Eric,
DeleteI have a quick question:
This is referring to step 7.
So I have POX running simple l2_forwarding. I ping from h1 to h3. I can see the flows being added from the Debug of POX CLI, but I don't see any flows being dumped when I use dpctl.
Here is the output:
mininet> h1 ping h3
PING 10.0.0.3 (10.0.0.3) 56(84) bytes of data.
64 bytes from 10.0.0.3: icmp_req=1 ttl=64 time=15.9 ms
POX> DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:03.3 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:01.1 -> 00:00:00:00:00:03.3
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:03.3 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:03.3 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:01.1 -> 00:00:00:00:00:03.3
mininet@mininet-vm:~/pox/pox$ sudo dpctl dump-flows tcp:127.0.0.1:6634
stats_reply (xid=0xf4171fd9): flags=none type=1(flow)
mininet@mininet-vm:~/pox/pox$ sudo ovs-ofctl dump-flows tcp:127.0.0.1:6634
NXST_FLOW reply (xid=0x4):
mininet@mininet-vm:~/pox/pox$ sudo ovs-ofctl dump-flows s1
NXST_FLOW reply (xid=0x4):
mininet@mininet-vm:~/pox/pox$ sudo ovs-ofctl dump-flows c0
ovs-ofctl: c0 is not a bridge or a socket
mininet> net
c0
s1 lo: s1-eth1:h1-eth0 s1-eth2:h2-eth0 s1-eth3:h3-eth0
h1 h1-eth0:s1-eth1
h2 h2-eth0:s1-eth2
h3 h3-eth0:s1-eth3
Any help would be greatly appreciated
Thanks,
Daniel
Hi Daniel, sounds like the flows is installed and you can see the ping flowing from the hosts, just not from the other monitoring tool. I think there are two things that we can try:
Delete1. if you enable x-forwarding as taught by the openflow tutorial, clear flow tables, and do tcpdump on all hosts, do you see what you would expect? (i.e. broadcast first packet then just h1-h3).
2. this is an assumption, but worth checking, prior to enable any constroller, the tutorial explained how to install flows manually by dpctl, it is probably worth checking again that dpctrl itself actually works.
I have also found that the VM / Mininet sometimes would give a hick up, for me it happens mostly when the VM is still running and I hibernate the machine, when I come back to it later there seems to be some small quirks and I have to restart mininet. I dont know if this just happens to me and my machine or not.
Let me know what you find out. tcpdump is really the next step for verification in my humble opinion.
HI Eric, now i want using Pox to control my topo. Example, when i connected to POX, i can ping h1,h2. But now i want h1 cant ping h2, can you tell me how can i do that by coding on POX, i dont know working on POX now.thanks!!!
DeleteHi Hai Pham, if I understand correctly, you have successfully implement whatever pre-made packages from POX to control your topology such as the dumb hub and L2 learning switch, now you would like to make 1 small tweak to make sure you understand how everything ties together?
DeleteIf the answer is yes, the short answer is I was in the same boat when I started to explore POX and my goal for the blog is to share my experience so others can build on top of it. So for example, in step 13, you can see the packet structure that was sent to the controller. So to do what you want to do, you can simply insert an if statement to look for (ip.src==10.0.0.2 and ip.dst==10.0.0.3) and of action to drop. Then everything besides those packets will pass, but the packets matching your description will drop, just like an access list. :)
Hope it helps.
Thanks very much, Eric can you send me your mail to my inbox, im studying at python code. If i have your mail, its very easily to contact you when i have a trouble.My mail: tuanhai.bk@gmail.com
Delete@Daniel, doing more work with OF / Mininet again. Wondering if you still cant see the flows in dpctl? Just tried it again with the new image with Mininet 2.0:
Delete1. Initial state with no flow:
mininet@mininet-vm:~$ date
Sat Apr 6 18:25:08 PDT 2013
mininet@mininet-vm:~$ dpctl dump-flows tcp:127.0.0.1:6634
stats_reply (xid=0x906ef4d2): flags=none type=1(flow)
mininet@mininet-vm:~$
mininet@mininet-vm:~/pox$ date
Sat Apr 6 18:25:19 PDT 2013
mininet@mininet-vm:~/pox$
2. Start POX in debug
mininet@mininet-vm:~/pox$ ./pox.py --verbose forwarding.l2_learning
POX 0.1.0 (betta) / Copyright 2011-2013 James McCauley, et al.
DEBUG:core:POX 0.1.0 (betta) going up...
DEBUG:core:Running on CPython (2.7.3/Sep 26 2012 21:51:14)
DEBUG:core:Platform is Linux-3.5.0-17-generic-x86_64-with-Ubuntu-12.10-quantal
INFO:core:POX 0.1.0 (betta) is up.
DEBUG:openflow.of_01:Listening on 0.0.0.0:6633
INFO:openflow.of_01:[00-00-00-00-00-01 1] connected
3. h1 poing to h2
mininet> h1 ping -c 3 h2
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
64 bytes from 10.0.0.2: icmp_req=1 ttl=64 time=6.88 ms
64 bytes from 10.0.0.2: icmp_req=2 ttl=64 time=39.4 ms
64 bytes from 10.0.0.2: icmp_req=3 ttl=64 time=0.456 ms
--- 10.0.0.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2005ms
rtt min/avg/max/mdev = 0.456/15.608/39.481/17.083 ms
mininet>
4. Flow gets installed:
DEBUG:forwarding.l2_learning:Connection [00-00-00-00-00-01 1]
DEBUG:forwarding.l2_learning:Port for 00:00:00:00:00:02 unknown -- flooding
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:02.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:01.1 -> 00:00:00:00:00:02.2
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:02.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:01.1 -> 00:00:00:00:00:02.2
5. dpctl shows flows:
ininet@mininet-vm:~$ dpctl dump-flows tcp:127.0.0.1:6634
stats_reply (xid=0xcdc223b6): flags=none type=1(flow)
cookie=0, duration_sec=6s, duration_nsec=220000000s, table_id=0, priority=65535, n_packets=3, n_bytes=294, idle_timeout=10,hard_timeout=30,icmp,in_port=2,dl_vlan=0xffff,dl_vlan_pcp=0x00,dl_src=00:00:00:00:00:02,dl_dst=00:00:00:00:00:01,nw_src=10.0.0.2,nw_dst=10.0.0.1,nw_tos=0x00,icmp_type=0,icmp_code=0,actions=output:1
cookie=0, duration_sec=5s, duration_nsec=185000000s, table_id=0, priority=65535, n_packets=2, n_bytes=196, idle_timeout=10,hard_timeout=30,icmp,in_port=1,dl_vlan=0xffff,dl_vlan_pcp=0x00,dl_src=00:00:00:00:00:01,dl_dst=00:00:00:00:00:02,nw_src=10.0.0.1,nw_dst=10.0.0.2,nw_tos=0x00,icmp_type=8,icmp_code=0,actions=output:2
cookie=0, duration_sec=1s, duration_nsec=181000000s, table_id=0, priority=65535, n_packets=1, n_bytes=42, idle_timeout=10,hard_timeout=30,arp,in_port=2,dl_vlan=0xffff,dl_vlan_pcp=0x00,dl_src=00:00:00:00:00:02,dl_dst=00:00:00:00:00:01,nw_src=10.0.0.2,nw_dst=10.0.0.1,nw_proto=1,actions=output:1
cookie=0, duration_sec=1s, duration_nsec=179000000s, table_id=0, priority=65535, n_packets=1, n_bytes=42, idle_timeout=10,hard_timeout=30,arp,in_port=1,dl_vlan=0xffff,dl_vlan_pcp=0x00,dl_src=00:00:00:00:00:01,dl_dst=00:00:00:00:00:02,nw_src=10.0.0.1,nw_dst=10.0.0.2,nw_proto=2,actions=output:2
mininet@mininet-vm:~$ date
Sat Apr 6 18:25:48 PDT 2013
mininet@mininet-vm:~$
Have you tried it again?
how to check flow entry is full or not
Deletein the pox controller
Deletethanks Eric!
ReplyDeleteHi Eric,
ReplyDeleteFirst of all it was very helpful in understanding it and cleared many doubts. I have one question. inside misc.of_tutorial.py file I wrote parser which uses Scapy.
def parseEachPacket(pkt):
if(pkt.haslayer(Ether)):
print "Ethernet::"
ether = pkt[Ether]
print " Src:: " + str(ether.src)
print " Dst:: " + str(ether.dst)
and i have imported required scapy packages. but when I run Error pop up:
AttributeError: 'ethernet' object has no attribute 'haslayer'
Any idea ? please help
Hi, I am new to Scapy and OF as well, just a fair warning.. :)
DeleteHow are you calling the function? Because the error seems to indicate that pkt in the function is not understood as a Scapy packet. Here is a simple program that I took from the Scapy tutorial with your function:
http://www.secdev.org/projects/scapy/doc/usage.html#simplistic-arp-monitor
** begin**
#!/usr/bin/env python
from scapy.all import *
def parsePacket(pkt):
if pkt.haslayer(Ether):
print "Found Ethernet packet"
print str(pkt[Ether].dst)
if __name__ == "__main__":
sniff(prn=parsePacket, store=0)
**end**
mininet@mininet-vm:~/pox$ sudo ./scapyTest.py
WARNING: No route found for IPv6 destination :: (no default route?)
Found Ethernet packet
:ac
Found Ethernet packet
:5a
....
The haslayer() function resides under packet.Packet:
In [1]: from scapy.all import *
WARNING: No route found for IPv6 destination :: (no default route?)
In [2]:
In [2]: packet.Packet.ha
packet.Packet.hashret packet.Packet.haslayer
In [2]:
So perhaps search for a way to make the pkt into a Scapy packet object would be the logical next step?
Also, wanted to mention that it almost seems easier to parse out the packet in OF then reconstruct the packet in Scapy. For example, in step 13, you already know the IP source, then you can use Scapy to construct a new packet with the same source:
Deletefrom scapy.all import IP
ipSource = packet.src
newPacket = IP(src=ipSource)
something like that might fit your needs?
i want to start up a dumb hub. however, when i type it out it gives me this "Module not found: of_sw_tutorial_myTest" how can i solve this?
ReplyDeleteBy default it searches the ~/ext directory, maybe check if the file is in that directory?
DeleteThis is very helpful, many thanks!
ReplyDeleteThanks for reading Yen. :)
DeleteThanks Eric for this tutorial. It is very helpful.
ReplyDeleteI've this question when I run:
./pox.py flow_stats l3_learning #### flow_stats.py from poxstuff/
with mn --controller remote
I've faced this error:
POX> INFO:openflow.of_01:[Con 1/1] Connected to 00-00-00-00-00-01
Task caused exception and was de-scheduled
Traceback (most recent call last):
File "/home/mininet/pox/pox/lib/recoco/recoco.py", line 276, in cycle
rv = t.execute()
File "/home/mininet/pox/pox/lib/recoco/recoco.py", line 94, in execute
return self.gen.send(v)
File "/home/mininet/pox/pox/lib/recoco/recoco.py", line 751, in run
rv = self._callback(*self._args,**self._kw)
File "/home/mininet/pox/ext/flow_stats.py", line 42, in _timer_func
connection.send(of.ofp_stats_request(body=of.ofp_port_stats_request()))
File "/home/mininet/pox/pox/openflow/of_01.py", line 572, in send
data = data.pack()
File "/home/mininet/pox/pox/openflow/libopenflow_01.py", line 2136, in pack
packed += struct.pack("!HH", self.type, self.flags)
error: cannot convert argument to integer
DEBUG:ext.flow_stats:FlowStatsReceived from 00-00-00-00-00-01: []
INFO:ext.flow_stats:Web traffic from 00-00-00-00-00-01: 0 bytes (0 packets) over 0 flows
Hi Ali, I am afraid that might be a bit out of my knowledge based on the information given. Is flow_stats your own code or one of the stock ones?
DeleteHave you tried the some of the email alias such as opeflow-discuss@ and mininet-discuss-requests@lists.stanford.edu? I find the list to be very helpful and knowledgeable, sometimes answered by creator of the tutorials/tools themselves.
Best of luck! :)
hey Eric,
ReplyDeleteGreat thanks for your great job!
It is a pity that there are no any references to this tutorial out there, I was having a tough time trying to understand the code before I got here. I think a small link to this tutorial in POX wiki and OpenFlow tutorial would be really helpful for newcomers.
Nevertheless, I hope I am not too late to a party! :)
I am trying to do a simple VLAN-translation function that would assign VLAN tags depending on given criteria.
It would be very interesting to hear your opinion and maybe suggestions on realization of such function.
Looking forward to hearing from you!
Regards,
Abe
Hi Abe, thank you for reading and the encouragement. :)
ReplyDeleteI am not sure I qualify to answer that VLAN-translation question. I think in today's implementation, it is already a bit complex to determine packet forwarding based not the existing fields; having the promise to swap headers is certainly possible, but would take more work. It is certainly fun to have this option, right? :)
Hi Eric,
ReplyDeleteactually I am new to python and POX.
This tutorial helped me so much to understand the basic functions of POX and how it is working.
would you mind tell me for understanding these codes in PYTHON, which reference would be useful?
and another question is about lazyhub.
I changed the dumbhub to lazyhub but it gives me erorr.
i will be thanksful if you can help me.
the error is as follow:
POX> INFO:openflow.of_01:[Con 1/1] Connected to 00-00-00-00-00-01
ERROR:core:Exception while handling OpenFlowNexus!PacketIn...
Traceback (most recent call last):
File "/home/mininet/pox/pox/lib/revent/revent.py", line 233, in raiseEventNoErrors
return self.raiseEvent(event, *args, **kw)
File "/home/mininet/pox/pox/lib/revent/revent.py", line 280, in raiseEvent
rv = event._invoke(handler, *args, **kw)
File "/home/mininet/pox/pox/lib/revent/revent.py", line 158, in _invoke
return handler(self, *args, **kw)
File "/home/mininet/pox/ext/of_sw_tutorial_myTest.py", line 33, in _handle_lazyhub_packetin
("ff:ff:ff:ff:ff:ff", event.ofp.in_port, "ff:ff:ff:ff:ff:ff", of.OFPF_ALL))
AttributeError: 'module' object has no attribute 'OFPF_ALL'
thanks a lot for providing this tutorial for others
Thank you for the kind words, Farzaneh. :)
Deletehi eric,
ReplyDeleteI have just started using pox
can you please guide how to handle "www" request if their is no DNS server?
thanks
Hi Gurjot, sorry for the late reply, I have been thinking about doing a Part 2 of the tutorial. The OpenFlow project has come very far and POX has a ton of new features as well. Let me bundle this in with other questions for the Part 2. Stay tuned! :)
DeleteI'm working with forwarding_l2.multi..
ReplyDelete1. Whether forwarding.l2_multi.py is for switch only?
2. That uses discovery.py to know the topology of the network and picks the shortest one, so where is the role of controller?
Waiting eagerly for your reply...
Thank you...
Bondankit07@gmail.com
Hi bondankit07, thank you for reading. I have been thinking about doing a part 2 for the tutorial with how different both OpenFlow and POX are now. I will probably bundle in the questions I have received and answer them together. Stay tuned! :)
Deletehi eric,
ReplyDeleteI am new in SDN. Can you please help me how to block ip addresses in mininet using pox controller?
thanks
Hi Divam, sorry for the late reply, I have been thinking about doing a part 2 of the tutorial with so much change in OpenFlow, SDN, and POX. I will bundle in the questions I have received and answer them together. Stay tuned! :)
DeleteHi Eric,
ReplyDeleteGreat job! It helps me a lot trying to understand the differences and limits between OpenFlow/POX/Python, I've been struggling a while the original tutorial.
Any clues on how to start designing and implementing a basic load balancer?
Thanks a lot!!
Hi Rous, thanks for reading! Yeah, when I first looked at POX the documentation were a bit lacking so I wrote this one after I bump my head many, many times. Glad it was helpful to you.
DeleteTo be honest, for basic load balancer, I will probably take a serious look at Google's project Seesaw, http://google-opensource.blogspot.com/2016/01/seesaw-scalable-and-robust-load.html. It is not Python, but I have heard good things about Go so it would be probably be a good opportunity to play with Go, I guess. :)
Hi Eric, great article for beginners It helped me a lot.
ReplyDeleteI've a confusion ho can I build something in a way that there're 6 host one switch and one controller, where 3 host (Hx) and other 3 host(Gx) can only ping to each other but not to others.
Like h1 ping g1 shouldn't ping but h1 ping h2 or g1 ping g2 should work.
Any help is appreciated
Thanks
Hi, thanks for reading. I think so.. you just in the rules for the traffic that you want to go thru but not the ones you don't. With OpenFlow that is the level of control you have. Have you tried it and it does not work somehow? :)
Deletehow to check network performance in pox or mininet.
ReplyDeletei want to check performance metrics like throughput, delay,packet loss etc.
Hi Mohammed, at least for BigSwitch controllers they report the switch stats at the controller level. Some of the switches also expose SNMP (I think). Do you have any particular vendor in mind?
DeleteThis is a great tutorial. Help me a lot.
ReplyDeleteCan anyone help me i want to drop packets from these two sources. but i am unable to run this. i am also new to python
def _handle_dumbhub_packetin(event):
packet = event.parsed
print packet
src_mac = packet.src
dst_mac = packet.dst
print src_mac
print dst_mac
if (packet.src == "00:00:00:00:00:01") or (packet.dst == "00:00:00:00:00:03"):
print "hello" # drop();
else:
send_packet(event, of.OFPP_ALL)
log.debug("Broadcasting %s.%i -> %s.%i"%(packet.src, event.ofp.in_port, packet.dst, of.OFPP_ALL))
if condition is not going to execute correctly.
Hi Sincer, just a note that I will look at this question when I spin up my OpenFlow lab again.. although I am not sure when I will get to it... :(
DeleteHi,
ReplyDeletecan you tell me how I can use POX with wireless interfaces ?
Yeah, if you use the OpenWRT54G Linksys with OpenFlow 1.0 load here is the instruction:
Deletehttp://archive.openflow.org/wk/index.php/OpenFlow_1.0_for_OpenWRT#LinkSys_WRT54GL
edit the /etc/config/wireless file, etc.
HI Eric,
ReplyDeleteCan you tell me how can I communicate between two SDN controllers?
Hi Sonal, from what I can tell the orchestration of controllers has always been fragmented and differs from implementer to implementer. The switch can always connect to multiple controllers, it is the automatic determining role of master/slave/dual-head that required some sort of controller-to-controller communication. This was defined in OF 1.2 and carried over to 1.3, https://www.opennetworking.org/images/stories/downloads/sdn-resources/onf-specifications/openflow/openflow-spec-v1.3.1.pdf but as you can see not much in terms of standardization.
DeleteSo... in short, if I were to do it, I would probably build my own using API by the controller itself instead of relying on the standard body. I think POX has not moved beyond OF 1.0, which was why I chose Ryu for my INE course. That would be my suggestion, use Ryu and see if you think the 1.3 spec fits your need, else use its included HTTP API to build your own.
Cheers and good luck. :)
This comment has been removed by the author.
ReplyDeleteHi Eric,
ReplyDeleteWhen I run sudo python ip_loadbalancer.py in my terminal I get :
File "ip_loadbalancer.py", line 23, in
from pox.core import core
ImportError: No module named pox.core
Please Help.
Hi Nitesh, core.py is still there by way of project code https://github.com/noxrepo/pox/blob/carp/pox/core.py and documentation, https://openflow.stanford.edu/display/ONL/POX+Wiki#POXWiki-ComponentsinPOX. If your ip_loadbalancer.py is your own code outside of the pox directory, then you need to add the POX directory to your Python module search path. This can be done using 'import sys' and 'sys.path.append("")'.
DeleteHi Eric,none of my pox programs are running.Pox is installed in /home/pox. All my codes are in some folders in /home/pox/pox. For example, /home/pox/pox/forwarding/hub.py. So what should I add in sys.path.append("") ???
DeleteHi Eric,
ReplyDeleteI am working on an overlay topology (where all switches are connected) and I have a controller connected to one of the switches (In band controller). In my controller, I need to know the IPs of the hosts (not only the switches). Is there a way to know that at the controller level? or should I import these IPs as an input to the controller?
Thanks in advance.
Can you please tell how to communicate between two pox controllers?
ReplyDeleteGreat Article please keep writing, It helps people alot who love to read good articles.
ReplyDeleteI've got plenty of interview questions over at algrim.co as well as 45 Selenium Interview Questions To Hire Or Get Hired With - Algrim.co
ReplyDeleteI have h1 and h2 as host and c1 as controller.
ReplyDeleteI want to change the POX controller so that if I ping between h1 and h3 it is h2 that receives the packets sent because h3 does not exist.this is my problem.I want to have ideas on that.
Thank you
hi Eric can you help me please, i created my topology which includes two mobiles staions and one access point and i want to see the messages received by the controller i followed the instructions but i have always this line [dump:00-00-00-00-00-01|4096] [ethernet][ipv4][udp][dhcp] i didn't understand why pox just work fine with the predefine cmd "sudo mn --wifi --topo linear,5 --controller=remote,ip=10.0.2.15,port=6633" but not if i create my file.py
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteGreat post, Please visit Courseya for Mysql free Courses
ReplyDeletethanks for reading,InterviewQueries
ReplyDeletethanks for this excellent and nice post, its a good one. i'm working on attack and normal traffic generation using scapy, i actually created the SDN topology but having issue with traffic generation and traffic statistic collection. could you please help me out?
ReplyDeleteAivivu - đại lý vé máy bay, tham khảo
ReplyDeletevé máy bay đi Mỹ tháng nào rẻ nhất
giá vé máy bay tết
vé máy bay đi Pháp bao nhiêu tiền
vé máy bay đi Anh bao nhiêu tiền
booking vé máy bay giá rẻ
vé máy bay đi San Francisco bao nhiêu tiền
thời gian bay từ Việt nam sang Los Angeles
combo marriott phú quốc
combo đi đà nẵng 3 ngày 2 đêm
Nice post and good to know about network automation nerds. Thanks for sharing.
ReplyDeletepsychometric test for employees
swift interview questions
Prepare to best MYSQL MCQS Questions
ReplyDeleteHey everyone, “India is a place where color is twice as bright. Pinks that make your eyes scorch, blues that you can drown in." Foreign citizens can come to India and see this country's beauty. But you don't know how to apply for an India e visa? You can get Indian tourist visa info via our Indian visas website.
ReplyDeleteVery interesting post and I want more updates from your blog. Thanks for your great efforts...
ReplyDeleteDivorce in Virginia with Child
Divorce in Virginia with Child
If you are planning to travel to Equatorial Guinea, you will need a visa. A visa is a special document that allows you to enter and stay in a country for a specific period of time. To apply for an Equatorial Guinea visa , you will need to fill out an application, pay a fee, and provide some personal information and supporting documents, such as your passport and an itinerary. The type of visa you will need will depend on the purpose of your visit, be it for tourism, business or study. Equatorial Guinea offers a range of visa types, including short-term visas and long-term visas for those planning to stay for an extended period of time. Before planning your trip to Equatorial Guinea it is important to check the visa requirements for your specific state.
ReplyDeleteWe value the information you gave. Thank you for providing this information. Top 30 Best Places To Live In The World In 2023. These sought-after destinations captivate with their unique charm, high quality of life, and promising opportunities. From the picturesque cities of Switzerland, known for their pristine landscapes and efficient infrastructure.
ReplyDeleteGreetings, fellow adventurers from the Dominican Republic! Dreaming of an Indian escapade? India Visa for Dominican Republic Citizens – you can conveniently apply for an e-Visa online. Whether it's for tourism, business, or medical purposes, the process is hassle-free. Just make sure your passport holds a validity of at least six months and carry a recent photo. Get set to embrace the enchanting wonders of India!
ReplyDeleteIt is truly remarkable and highly appreciated that you are willing to impart knowledge and insight. I have greatly appreciated your guidance. Ghanaian adventurers! Planning an Indian getaway? India Visa for Ghana Citizens. you can apply for an e-Visa online. Whether for tourism, business, or medical purposes, the process is straightforward. Just ensure your passport has six months' validity and a recent photo. Get ready to experience the vibrant culture of India!
ReplyDeleteHello! Your content captivates and informs effectively. Eagerly anticipating further insights from you in the times ahead! Planning a trip? Discover the essentials of Egypt visa for Bulgaria citizens. Ensure a seamless journey by understanding the Egypt visa requirements for a memorable experience.
ReplyDeleteGood morning everyone, Many people also ask about , 5 Things to Know Before Traveling to Turkey ? Planning a trip to Turkey? This captivating country offers a blend of rich history, vibrant culture, and stunning landscapes. To ensure a smooth and enjoyable journey, here are five essential things you should know before you go.
ReplyDeleteHello! I wanted to express my gratitude for the valuable information shared on your blog. Your dedication to providing in-depth insights is truly commendable. Are you a Malaysian citizen planning a trip to Saudi Arabia? Check out Saudi Arabia Visa for Malaysia Citizens and explore the visa requirements, application process, and essential tips for a smooth travel experience.
ReplyDeleteHlo sir, Discover the convenience and simplicity of the e-Visa system for Azerbaijan. With the e visa of Azerbaijan, travelers can easily obtain their travel authorization online, making their journey to this beautiful country hassle-free and efficient.
ReplyDelete"This post is a gem! Your effort in sharing such valuable information is truly appreciated." Solomon Islands citizens can obtain an Indian visa for travel, business, or medical reasons. India Visa for Solomon Islands Citizens. Apply online or at designated centers. Ensure you have the necessary documents, pay the fees, and follow processing guidelines, which may vary. Be sure to check eligibility before applying and look forward to a memorable trip to India!
ReplyDeleteObtaining a Turkey visa from Palestine is a crucial step for Palestinian travelers seeking to explore Turkey's rich cultural heritage, breathtaking landscapes, and historical treasures. This process ensures that Palestinian passport holders can embark on their Turkish adventures with ease. Travelers from Palestine can apply for their visas through the Turkish consulate or embassy in Palestine, adhering to the required documentation and procedures. Alternatively, the convenient e-Visa system offers a streamlined and user-friendly application process, simplifying travel planning to Turkey. With the appropriate visa, Palestinian adventurers can immerse themselves in the enchanting experiences that Turkey has to offer, creating lasting memories.
ReplyDeleteHello! Great post, I found it very informative and helpful. Looking forward to more content! Getting an Azerbaijan visa for Indians is straightforward. Ensure a hassle-free visit to this beautiful country by checking the latest requirements and application process for Azerbaijan visa for Indians.
ReplyDeleteHello everyone, How are you all of you. Exploring India's wonders has never been easier. Learn about the Indian Visa cost and plan your unforgettable journey to this diverse and captivating country without breaking the bank.
ReplyDeleteExcellent post! Concise, informative, and highly valuable content. Well done! Requirements to enter Saudi Arabia include a valid visa, passport, and adherence to their specific regulations, such as dress code and customs guidelines.
ReplyDeleteIt's wonderful that you're willing to share these with us. I'm enthusiastic about joining the event in the future, as this topic resonates with me. Diwali celebrations across the globe
ReplyDeleteI genuinely appreciate your inclination to share these with us. In the days ahead, I'll be actively participating in the event, as this subject matter deeply interests me.Here is the important news morocco visa for Indians. Are you excited to know more about it? I am sharing this here because everyone benefits from it.
ReplyDeleteYour blog consistently delivers valuable insights! The way you articulate complex topics is truly impressive. Your dedication to quality content is appreciated. Keep up the excellent work! Explore the convenient Saudi on Arrival visa for Indian Passport holders
ReplyDeleteWithin your blog post, you embark on a captivating literary exploration, effortlessly traversing the realms of intellect and emotion. The harmonious blend of profound wisdom and relatable anecdotes is truly noteworthy. Do you need a visa for turkey from Australia Yes, if you're an Australian citizen planning to visit Turkey, you'll need to apply for an visa before your trip. This electronic visa can be conveniently obtained online, streamlining the entry process for tourists and ensuring compliance with Turkey's travel regulations.
ReplyDeleteYour perspectives are genuinely refreshing! It's intriguing how you delve into various dimensions of thought. Your articulate expression effortlessly encapsulates the essence of the discourse, making it an engaging read, Unlocking Bahrain's wonders is now a breeze with the convenience of on-arrival visa! Explore its treasures without breaking the bank. Embrace adventure, culture, and hospitality without worrying about bahrain visa cost on arrival
ReplyDeleteComposing blog posts is part of my regular routine, and I genuinely admire the quality of your content. This captivating piece has sparked my interest. I've ensured to save your site in my bookmarks and keep myself informed with fresh insights, typically on a weekly basis. Additionally, I've subscribed to your RSS feed. Does a UK citizen need a visa for Egypt? Yes, a UK citizen needs a visa for Egypt. Visa requirements vary based on the purpose and duration of stay. Conduct thorough research for accurate information.
ReplyDeleteThis comment has been removed by the author.
ReplyDeletePlanning a trip to Kenya? Wondering about Kenya eta application cost. Look no further! Our comprehensive guide covers everything you need to know about Kenya ETA application fees. From processing costs to additional charges, we've got you covered. Start your journey hassle-free
ReplyDeleteI’m deeply appreciative of the effort you put into this outstanding blog. Every part was sincerely valued, and I’ve bookmarked it to keep up with your latest posts. Your expertise in explaining complex concepts with such clarity and depth is truly impressive. what is e visa in Ethiopia An e-Visa for Ethiopia is an electronic visa system that allows travelers to apply for and receive their visa online before entering the country. It streamlines the visa application process, making it more convenient and efficient for visitors to obtain travel authorization.
ReplyDelete