I’m starting a new post series to walk you through building a private cloud. This post is the first one in the series and will be just an introduction to private clouds.
In the next posts I will discuss collecting the requirements, building the network and network services, building the storage, designing the server infrastructure, and putting the orchestration layer on top of that.
Why Private Cloud?
There are number of public cloud providers out there that have good and affordable offerings. However customers sometimes prefer private cloud over public cloud for specific reasons or requirements they need to meet. Here are some the reasons that come to mind:
Regulatory Compliance: Some companies have have to comply with certain regulations and therefore have to keep their customer data or their own data on dedicated/isolated infrastructure behind their firewall. In this case the multi-tenant public cloud model won’t be a good fit for them.
Control: Some companies require more control over the cloud infrastructure than what public cloud providers usually offer or they might have a specific need to integrate a solution/product from certain vendor. Most public cloud providers offer standard services that may not be customized or tweaked easily.
Avoid Lock-in: I’m sure you heard this before. People don’t like to put all of their eggs in one basket nor like to heavily rely on a specific vendor/provider. By building their own cloud in house companies can have the freedom of choosing multiple vendors and can also leverage open source technology if they wish.
Cost: This might be a bit of surprise to you but depending on how much the monthly bill coming from the public cloud provider is, operating a private cloud might become a cheaper option. This is especially true if you are using the cloud to run production workloads that need to be up 24/7 and cannot be turned off while not in use.
Barriers to Private Cloud Deployments:
Lack of Technical Resources: Designing, implementing, and supporting a private cloud require having the right IT resources in house. Some companies just don’t have that.
Time to Market: Getting a private cloud up and running is at least 6-12 month long process. Some companies have to respond to market pressure quickly and simply don’t have the luxury of spending a year to build a cloud in house.
Cost: Building and operating a private cloud is expensive and it makes more sense in some cases to leverage a public cloud offering than spending that CapEx up front.
You have the capital and technical talent and want to build a private cloud? Great. In the next post I will walk you through the process of collecting the requirements and making high-level design decisions before we get into building the infrastructure itself. Stay tuned.
Make sure that you either subscribe to my blog or follow me on Twitter to get notified when I add a new content.
This post assumes a working knowledge of Locator/ ID Separation Protocol (LISP), you may first want to review Part 1 before reading through this post.
In Part 1 we created two LISP sites and enabled connectivity between these two sites. We also enabled VM mobility and showed how you can migrate a VM to the cloud without changing its IP address, mask, or default gateway.
In this post I will expand on Part 1 and enable connectivity between non-LISP and LISP sites. As shown in the diagram below I have a branch office (non-LISP site) which needs to reach the VM (IP address 10.122.139.91) in the cloud.
To do so we have three design options…
Option 1 is to enable LISP on the branch edge router. In this design traffic leaving the branch will be redirected by the branch edge router to the cloud however this option is costly and requires deploying a router with LISP support on each branch router.
Option 2 is to deploy a LISP proxy router in the service provider WAN to intercept traffic and re-direct it to cloud. This option is less expensive than option 1 but requires routing all traffic thru the proxy router which could cause this router to become a choke point.
Option 3 is to configure the already LISP-enabled router in the data center as proxy which will redirect traffic to the cloud. This is the least expensive option but obviously does not provide optimal routing as you need to backhaul traffic to the data center first, however this might be acceptable design when local internet access is not available at the branch and internet-bound traffic needs to traverse the data center anyway or when the customer wants the traffic from the branch to traverse the data center firewall first for policy enforcement.
In this post I will show you what you need to do to enable option 3.
To add the proxy functionality to the design, first we need to configure CSR1 as Proxy Tunnel Router (PxTR) which tells CSR1 to intercept the traffic coming from any non-LISP site and redirect it to the cloud:
CSR_1#sh run | i ipv4
no ipv4 itr
ipv4 proxy-etr
ipv4 proxy-itr 1.1.1.1
Now if I run a show command I can see that CSR1 is configured as PxTR:
CSR_1#sh ip lisp
Instance ID: 0
Router-lisp ID: 0
Locator table: default
EID table: default
Ingress Tunnel Router (ITR): disabled
Egress Tunnel Router (ETR): enabled
Proxy-ITR Router (PITR): enabled RLOCs: 1.1.1.1
Proxy-ETR Router (PETR): enabled
The second step is to configure CSR3 to use CSR1 as its default proxy ETR (PETR). This is necessary as CSR3 is not going to know how to get back to the source 2.2.2.2 since this prefix is not a LISP entry and does not exist in its cache table.
CSR_3# show run | b lisp
router lisp
ipv4 use-petr 1.1.1.1
Now let’s run a ping test from R2 (2.2.2.2) to see if we can reach the VM:
R2#ping 10.122.139.92 source 2.2.2.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.122.139.92, timeout is 2 seconds:
Packet sent with a source address of 2.2.2.2
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 48/51/63 ms
Comment below if you have any questions. Also please help me spread out the word if you like this post and share with your network on Twitter/Facebook/Linkedin.
There are times when you want to leverage public cloud but want to keep some workloads in your data center. You can do that with Hybrid Cloud.
According to Gartner the most important use case for Hybrid Cloud is cloud bursting. That’s running the workloads internally in the data center and bursting to a public cloud during peak times. The other common use cases for hybrid cloud is disaster recovery where you migrate the workloads to a public cloud during disasters or maintenance windows.
Migrating a workload to the cloud is time consuming especially if you have to re-IP all the VMs.
In this post I will show how to migrate an application to the cloud without changing its IP address, mask, default gateway, or security policies. This makes the migration easier especially if you are migrating the workload temporarily and do not want to change the IP addresses or DNS settings.
Why I Like This Solution
You can certainly enable workload mobility between two data centers using a Layer 2 extension technology such as VPLS or Cisco OTV. However I prefer LISP over any L2 extension simply because it’s based on Layer 3 and therefore does not exntend the broadcast/failure domain like L2 extensions and you get all the benefits and scale of L3 routing.
I also think that using LISP is better than relying on any cloud extension technology like Cisco Nexus 1000v InterCloud or Verizon CloudSwitch because those technologies add a shim layer to allow the VM to run in a cloud provider environment and that degrades the VM performance. With this LISP design the VM runs natively on the hypervisor and behaves like any VM local to the cloud provider environment.
Prerequisites
– To get this solution to work to you need two Cisco routers that support Locator/ID Separation Protocol (LISP). The data center router can be either virtual or physical while the cloud provider router has to be virtual. I will be using here two Cisco CSR 1000v routers running IOS XE 3.12S for this solution.
– I will briefly explain how LISP works along the way and will be focusing only on the required components for this design. You can find more information on LISP on Cisco.com
– I will be using Verizon Terremark as my cloud provider. You may use other providers like Microsoft Azure or AWS if you wish.
Before The Migration
The diagram above shows a VM running in a data center/private cloud with IP address 10.122.139.92. The default gateway for this VM is currently CSR-1 and from the outside DNS resolves to the VM public IP address which is part of the data center addressing scheme. Now because CSR-1 is a LISP enabled router and because it’s also acting as a LISP map-server/map-resolver, it will cache the VM IP address in its database so it can inform other LISP routers about the VM location when asked.
In the cloud provider, CSR-3 is also a LISP enabled router and configured to query CSR-1 for any hosts trying to reach. So if CSR-3 receives any traffic destined for the VM it queries CSR-1 for the location of that VM. CSR-1 replies back to CSR-3 and tells it to encapsulate and send all traffic designed for the VM to him because the VM is located on its LAN.
The output below shows that the VM is located on CSR-1 local subnet at the moment:
CSR_1#sh ip lisp forwarding eid local
Prefix
10.122.139.92/32
The next output below shows that CSR-1 (1.1.1.1) is currently the locator for the VM:
On CSR-3 if I issue “where is the VM” command I see that CSR-1 (1.1.1.1) is currently the locator:
CSR_3#lig 10.122.139.92
Mapping information for EID 10.122.139.92 from 10.0.20.1 with RTT 49 msecs
10.122.139.92/32, uptime: 00:02:28, expires: 23:59:59, via map-reply, complete
Locator Uptime State Pri/Wgt
1.1.1.1 00:02:28 up 1/100
That’s good. Both routers are in sync and know that the VM is located in the data center. Let’s now migrate this bad boy to the public cloud.
After the Migration
Now the VM has migrated (cold migration) to public cloud as shown in the diagram above. When the VM boots up of the first time it will ARP for its default gateway (which is CSR-3 in this case). When CSR-3 detects that the VM is now on its LAN it registers it with the map-server (CSR-1) and declares itself as the new locator for that VM. Now the DNS mapping for the VM has not changed and the outside world still thinks that the VM resides in the enterprise data center. So when CSR-1 receives traffic destined for that VM it does a lookup in its database, encapsulates the traffic, and sends it to the new locator (CSR-3) which in turn decapsualtes the traffic and sends it off to the VM.
Now back to the router CLI. If I issue the “where is the VM now” command again, I see that CSR-3 (3.3.3.3) is now the locator:
CSR_1#lig 10.122.139.92
Mapping information for EID 10.122.139.92 from 10.0.20.3 with RTT 48 msecs
10.122.139.92/32, uptime: 00:02:27, expires: 23:59:59, via map-reply, complete
Locator Uptime State Pri/Wgt
3.3.3.3 00:02:27 up 1/100
If I ping the VM from CSR-1 sourcing my packets from the LAN interface, the ping is successful:
CSR_1#ping 10.122.139.92 source loopback 10
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.122.139.92, timeout is 2 seconds:
Packet sent with a source address of 10.10.10.1
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 47/47/48 ms
Conclusion
LISP allows migrating workloads to the cloud without relying on L2 extensions. Use this design if you are migrating your workload temporarily. If you are migrating the workload permanently you might want to consider adding a LISP router at the WAN edge or at the branch office to redirect user traffic to the public cloud provider before its enters the enterprise data center.
Ping me if you want the full configurations.
Part 2 of this post is coming soon, make sure to subscribe to the blog or follow me on Twitter to get notified when new content is added.
I’m in the process of putting together a new course called Introduction to Cloud Computing. It will be delivered via email so you can work at your own pace and above all it’s free. If you are interested go ahead and subscribe below to receive the weekly emails from me once the course starts. The course starts in October and runs for 6 weeks. You may unsubscribe at any time.
Here is what you will learn in this course:
Week 1 – The Basics of Cloud Computing
Week 2 – Is Cloud Right For Your Business?
Week 3 – Cloud Models: Private, Public, and Hybrid Cloud
One of the common use cases for the CSR 1000v is a secure VPN gateway in the cloud to terminate VPN tunnels. So if you run some applications in the cloud and you want to allow your branch offices to access those application over a secure network, you can run IPsec tunnels between those branch offices and the CSR 1000v in the cloud. From performance perspective this would work much better than back hauling the traffic to the data center (HQ) and then to the cloud.
In this post I will show you how to set up a VPN tunnel between Cisco CSR 1000v and a branch office router using Cisco Easy VPN. You may also use standard IPSec VPN but in my configurations I use Easy VPN because it’s, well, easier 🙂
The following configurations are part of a demo I gave at Cisco Live 2013 in Orlando where the CSR 1000v was the VPN server and the branch server (Cisco 2900 in this case) was the VPN client.
You will also need to open UDP port 500 and IP protocol ID 50 (ESP) on all firewalls sitting between the CSR 1000v and the branch router for the IPSec tunnel to be established successfully. Additionally depending on your design you may need to configure NAT.
In my setup, I’m using:
Cisco CSR 1000v with IOS XE 3.9x running in Verizon Terremark Enterprise Cloud
Cisco 2900 running IOS 153.2.T (branch office)
Here are the configs of the CSR 1000v (VPN Server). Only relevant configs are shown:
CSR-1#sh run hostname CSR-1 ! aaa new-model ! ! aaa authentication login hw-client-groupname local aaa authorization network hw-client-groupname local ! ! aaa session-id common ! ! username myusername password 0 mypassword ! ! crypto isakmp policy 1 encr 3des authentication pre-share group 2 crypto isakmp client configuration address-pool local dynpool ! crypto isakmp client configuration group hw-client-groupname key hw-client-password dns 8.8.8.8 domain domain.com pool dynpool save-password ! ! crypto ipsec transform-set transform-1 esp-3des esp-sha-hmac mode tunnel ! ! ! crypto dynamic-map dynmap 1 set transform-set transform-1 reverse-route ! ! crypto map dynmap client authentication list hw-client-groupname crypto map dynmap isakmp authorization list hw-client-groupname crypto map dynmap client configuration address respond crypto map dynmap 1 ipsec-isakmp dynamic dynmap ! ! ! interface GigabitEthernet2 description WAN ip address 10.22.36.71 255.255.255.224 negotiation auto crypto map dynmap ! interface GigabitEthernet3 description LAN ip address 10.22.39.91 255.255.255.240 negotiation auto ! ip local pool dynpool 192.168.1.1 ! ! ip access-list extended split_t permit ip 10.22.39.0 0.0.0.255 any ! end
Here are the configs for the Cisco 2900 router (VPN client).
no aaa new-model ! ip cef ! ! username vpntest password 0 vpntest ! ! crypto ipsec client ezvpn hw-client connect auto group hw-client-groupname key hw-client-password mode client peer 10.22.36.71 username myusername password mypassword xauth userid mode local ! ! interface GigabitEthernet0/0 description WAN ip address 10.35.120.104 255.255.255.0 duplex auto speed auto crypto ipsec client ezvpn hw-client ! interface GigabitEthernet0/1 description LAN ip address 172.16.4.102 255.255.255.0 duplex auto speed auto crypto ipsec client ezvpn hw-client inside ! ip route 0.0.0.0 0.0.0.0 10.35.120.1 ! end
The Cisco Cloud Services Router (CSR) 1000v is Cisco’s first virtual router and it runs as a VM on x86 virtualized server. The CSR 1000v runs Cisco IOS XE and brings many benefits to a cloud environment where it can operate as a secure VPN gateway to terminate site-to-site IPSEC tunnels. Other use cases of the CSR 1000v include MPLS WAN termination and control & traffic redirection. You can find more information n this virtual router on the Cisco CSR 1000v page.
In this post I’m going to show you how to install the CSR 1000v on the Verizon Terremark eCloud. If you are not familiar with the Verizon Terremark cloud console, you can go here for some video tutorials.
Download A Free Trial Image
Go to the Cisco Download Center and log in using your Cisco credential. Select the latest software release from the left pane and then download the .iso package from the right pane to your desktop. Don’t download the OVA or BIN packages. This free trial should be good up to 6 months.
You can purchase a license from Cisco and upgrade to a full version after you install the CSR.
Create the VM
Go over to the Verizon Terremark Enterprise Cloud (eClould) page and log in using your credential.
The first step here is to create a blank server. A blank server is just a container inside the VMware environment that will hold the Cisco IOS later. So go to Devices -> Create Server -> Blank Server to launch the wizard.
On the next screen select Linux for the OS Family, Other 2.6x Linux (64-bit) for the Version. Give the server a name and a description (optional). Then choose an INT network andchoose in which Row and Group you want to place the router.
On the next screen choose 4 or more for the Processors and 4096 MB or more for Memory.
On the next screen enter 8 for the Disk size or choose a Detached Disk that’s at least 8GB. Click Next.
From the next window you can optionally assign tags or just click Next if you don’t want any tag.
On the next window review your configurations, check the Agreement box, and click Deploy. Give it about 2 minutes and the server should be ready. If you go to the row and group where you placed the router you should see a greyed out icon (powered off) there.
If you recall we attached only one network to the server earlier, however the CSR requires a minimum of 3 interfaces to run so next we are going to create two additional networks and attach them to the blank server. Right click on the server and go to Configure to launch the wizard.
On the next screen choose Configure the Server and click Next.
On the next screen click on Network Settings from the top. At this point you should see only one INT network under the Current Connection. Click Add a Connection from the top right to add a new DMZ Network.
Go click again on Add a Connection and this time choose an INT network (INT_XXX). Choose a unique INT network different from the one you have used when you created the blank server. Click Save.
Now the VM will take a minute or so to reconfigure itself.
Once the wheel stops spinning and VM is ready, right click on it and go to Power On. Powering on this VM should take a minute or so and while the machine is powering on an icon with gear spinning is presented. The greyed out icon should turn into an icon with “blue light” once the server is completely powered on.
Console to the VM
In order to console to the blank server you must VPN first and use the VMware remote console plugin in Firefox or Internet Explorer to connect to the server for the first time. So for the following steps make sure you are using either Firefox or IE. Once the CSR is up and running you can switch back to your favorite browser or use a Telnet/SSH client to connect it directly instead of using the console.
Click on the VPN Connect link from the top of the page. This will install the Cisco AnyConnect Mobility Client on your machine if you don’t already have it and connect you via secure SSL VPN to eClould. If the browser prompts you to install any Java or ActiveX plugin during this step make sure to accept.
Now click on the router icon and then click on Connect from the bottom pane. This will start installing the VMware remote console plugin if it’s not already installed on your machine and will connect you to the server console to install the CSR.
By now you should see “Operating system not found” message on the console screen. Go to Devices-> CD/DVD drive -> Connect to Disk Image File ISO. This will open a new window for you to locate the ISO package.
Select the ISO package from your local machine and click OK. The system will start uncompressing/unzipping the package and installing software. Depending on the file size and your network speed this process may take up to 2 or 3 hours. Come back in 3 hours to check on the install process. If the console session times out, reconnect again as you did previously.
Once the CSR is successfully installed, you should see the traditional Cisco router prompt and you can then log in and start configuring it.
Assigning IP Addresses
In this section we will assign IP addresses to the CSR 1000v interfaces. When we created the VM we added 3 interfaces for the CSR 1000v to use (INT_XXX, DMZ_XXX, and INT_XXX). Each CSR interface will map to a logical vNIC assigned by the VMware hypervisor. The vNIC in turn is mapped to a physical MAC address.
The following steps are very important and you should not begin configuring any features on the CSR 1000v before you execute all of them.
First we need to find the CSR interface to vNIC mappings so go ahead and create the following table on a piece of paper. You will be populating its fields along the way:
NIC ID
Network
MAC
CSR Interface
IP/Mask
Default Gateway
1 (eth0)
INT
2 (eth1)
DMZ
3 (eth2)
INT
Log in to the CSR console and issue the following command:
show platform software vnic-if interface-mapping
You should see three interfaces with their vNIC and MAC addresses mapping. If you are running IOS XE 3.10S or earlier the first interface from the top should be GigabitEthernet0 and it’s the management interface. On the other hand if you are running 3.11S or higher the first interface is usually GigabitEthernet1. Start populating the table you have created n the previous step with the information from the command you just executed (IP information will come in the next steps) .
Go back to the eClould portal, click on the router and go then to Administrative Tasks -> Manage PIs
From the Networks section, click on the drop-down menu and select your DMZ network. Then from the left pane Available IPs select one IP address and click on the rightgreen arrow to assign it to the server. Click Save.
Repeat the previous step to assign IP addresses for the INT (internal) interfaces.
Go back to the table and fill out the IP addresses, mask, gateway sections before you move on.
Now that you have the IP addresses, masks, and interface mappings start configuring the DMZ and INT interfaces of the CSR by issuing the following commands for each interface:
config t int gX ip add <ip> <mask> description <Outside or LAN interface>
Configure A Default Route
The final step here is to assign a default route so that the CSR can reach the internet. The CSR must use the DMZ default gateway to reach the internet because only the DMZ network can reach the internet because the INT networks are not visible from the outside. So copy from the table you have created the default gateway IP address for the DMZ network.
Go back to the CSR console and issue the following commands. Make sure that you replace the text in italic with the IP address of the default gateway
config t
ip route 0.0.0.0 0.0.0.0 <DMZ_Default_gateway>
end
write mem
Try now pining the default gateway, you should get a response:
ping <DMZ_default_gateway>
At this point, the CSR should be able to reach the internet. Issue the following ping to make sure:
ping 8.8.8.8
If you don’t get a response then something is wrong and you should revisit the previous steps.
