Showing posts with label Public Cloud. Show all posts
Showing posts with label Public Cloud. Show all posts

Sunday, August 23, 2015

Amazon Web Services: Cloudformation


In this blog, we will discuss Cloudformation.
cloudformation helps in codifying creation of stack of resources.

What are stack of resources? A stack could be:

  • ELB
  • Auto scaling group
  • EC2
  • RDS (Database)
  • All connections between them

There are many benefits of cloudformation, namely:

  1. Your infrastructure as a code
  2. can be version controlled
  3. Modularization ( templates can contain other templates)
  4. Enforce one way to deploy
  5. Cost nothing
Signing up with Cloudformation means signing up with AWS services that Cloudformation can create

With Cloudformation you can setup billing alarms. To learn how to setup billing alarms go to

There are 200-300 pre-packaged amazon templates available. The templates are JSON based. The templates can accespt run-time parameter [Instance type (small for test), Kep pair (prod vs Test)]
Templates can also update running stack
Templates have 7 section
  • version
  • description
  • parameters
  • mapping
  • resources
  • properties
  • output
The link for templates is

Friday, August 21, 2015

Amazon Web Services: Auto scaling and Cloudwatch


 In this blog, we will discuss Amazon Web Services: Auto scaling and Cloudwatch

The question is : What does Auto scaling provides?  The answer is:

  • Linear Horizontal Scaling 
  • HA

Second question is What Cloud watch enables? Cloud watch enables:

  • monitoring and alerting for resources
  • Alarms Can fire auto scaling policies, SNS notifications

 auto scaling, ELB and Cloud watch makes up what Amazon calls Triangle services

AUTO SCALING SET UP

To set up Auto scaling, we need to do the following things:

  • Create ELB( Its recommended but not necessary)
  • Create Instances in 2 or more AZs ( Its recommended but not necessary
              a) Create Launch Config: Launch config consists of
                          1) EC2 instance launch profile
                          2) Image ID (AMI)
                          3) Instance Type ( e.g t1.micro or m1.small)
                          4) key pair ( recommended)
                          5) Security Group
                          6) Name

  • Create Auto scaling Groups: In a auto scaling group you have AZs, Max or Min servers etc
  • Create Auto scaling Policy: The policy consists of buttons that an API/Scheduled/ cloudwatch can push
  • Create Cloud watch Alarms: Which can fire policies




CLOUD WATCH


Now we will briefly discuss cloud watch. So what does Cloud watch exactly do? Well, the answer is
  1. Monitors/Alarms on resources
  2. Basic Monitors provided free with most basic services ( EC2, ELB, SQS..etc)
  3. Alarms can fire auto scaling policies or SNS notifications
  4. cloud watch is statistics in , statistics out. below is an example














Cloud Watch alarms trigger when metric statistics goes in( or out  ) out of state for specific time

Hope it clears the functionality of cloud watch




Thursday, August 20, 2015

Amazon Web Services- Elastic Load balancer



In this Topic we will discuss Amazon Web services ELB or Elastic Load Balancers

So first question is : What does Load balancers Do?  The answer is:

  •  Send Traffic to many Back-end Servers
  • Attempt to evenly distribute load
  • Allows yours service to be elastic
  • It allows for Horizontal Scaling
  • Grow and shrink on demand
However,

Horizontal Scaling by the ELB in the cloud gives us:
  • Near Infinite Scaling
  • Huge cost savings
  • High Availability
  • Scaling By Tier
So what is Scaling by Tier? 

We can scale up or down web tier independent of APP Tier. This sis called scaling by tier

ELB Auto-scales ( Adds more IPs with increased traffic to the DNS)

ELB can also Terminate SSL, encrypt SSL to the back-end Machines

ELB also supports X-Forwarded-FOR

ELB has deep Integration with Auto Scaling and Cloud watch

Integration with Auto Scaling and Cloud watch

ELB can detect unhealthy Instances
  • Unhealthy by TCP or HTTP check
  • It automaticvally removes them dfrom scaling

However, Cloudwatch can add more instances behind ELB
  • Based on ELB metrics ( RequestCount, Latency and More)
  • Based on Instance Metrics (CPU. I/O, etc)


Remember,
  • Load balanced is first by Availability Zone
  • Then across instance in availability zone
  • ELB cost is static
  • ELB scaling is always reactive

Sunday, October 19, 2014

Microsoft Private cloud part 2: VMM2012 Installation Requirements




VMM Management Server
The general requirements for the VMM management server include the following:
  •            The VMM management server must be a member of an Active Directory domain.
  •         The name of the VMM management server can have a maximum length of 15 characters.
  •       Startup memory should be 3 GB if the VMM management server runs on a Hyper-V R2 SP1 host with dynamic memory enabled.
  •     Running the VMM management server on a failover cluster requires Windows Server 2008 R2 Enterprise or Datacenter edition.


Recommended Hardware Requirements for the VMM Management Server
Hardware Component             Up to 150 Hosts                           More than 150 Hosts
Processor                                  Dual x64 CPU, 2.8 GHz                    Dual x64 CPU, 3.6 GHz
RAM                                                  4 GB                                                       8 GB
Disk                                     40 GB (without local VMM database)     50 GB (without local VMM database)
                 150 GB (with local, full version of Microsoft SQL Server) 150 GB (with local, full version of Microsoft SQL Server)

Software Requirements for the VMM Management Server
Software Requirement                                                 
A supported operating system                      
Windows Remote Management  (WinRM) 2.0
Microsoft .NET 3.5 SP1 T
Windows Automated  Installation Kit (WAIK) for Windows 7



VMM Console
Installing a VMM management server automatically includes installing the VMM console. The VMM management server must be a member of an Active Directory domain. Table 4.3 lists recommended hardware requirements for the VMM
console.

.
Recommended Hardware Requirements for the VMM Console
Hardware Component                    Up to 150 Hosts                                                   More than 150 Hosts
Processor                                             Dual x86 or x64 CPU, 1.0 GHz                            Dual x86 or x64 CPU, 2.0 GHz
RAM                                                                1 GB                                                                     2 GB
Disk                                                                 2 GB                                                                     4 GB


VMM Self-Service Portal
The general requirements for the VMM self-service portal include the following:
·         Installation must be on a separate (virtual) machine.
·         The self-service portal cannot be installed on a domain controller.
·         Client computers accessing the self-service portal must use Internet Explorer 8 or later.
·         The requirements for the VMM self-service portal depend on the number of simultaneous connections to the web server.



Recommended Hardware Requirements for the VMM Self-Service Portal

Hardware Component                       10 Connections or Fewer                         More than 10 Connections
Processor                                                  Single x64 CPU, 2.8 GHz                            Single x64 CPU, 2.8 GHz
RAM                                                                       2 GB                                                         8 GB

Disk                                                                      20 GB                                                       40 GB


VMM Database

The general requirements for the VMM database portal include the following:
·         The VMM database server must be a member of an Active Directory domain. If that domain differs from the one containing the VMM server, a two-way trust relationship must exist between the two domains.
·         The name of the server cannot exceed 15 characters.
·         The SQL instance needs to be case-insensitive with collation type SQL_Latin1_General_CP1_CI_AS. Types other than Latin1 are supported as long as the collation type is CP1_CI_AS.
·         The required server roles for the VMM admin account on the VMM database are dbcreator, process admin, and security admin.
·         Database role membership should be db_owner.


   Recommended Hardware Requirements for the VMM Database Server

  Hardware Component           Up to 150 Hosts                                                   More than 150 Hosts
  Processor                                            Dual x64 CPU, 2.8 GHz or faster                  Dual x64 CPU, 2.8 GHz or faster 
  RAM                                                    4 GB                                                              8 GB
  Disk                                                     150 GB                                                          200 GB

  Software Requirements for the VMM Database Server

Software                                            Requirement Notes
  A supported operating system          A full installation of Windows Server 2008 R2 (Standard, Enterprise, or
                                                                Datacenter) with SP1 or later. If SQL Server is clustered, only Enterprise and
                                                                 Datacenter are supported.
  SQL Server versions                           SQL Server 2008 R2 Datacenter (32-bit and 64-bit) with SP1 or later
                                                          SQL Server 2008 R2 Enterprise (32-bit and 64-bit) with SP1 or later
  SQL Server 2008 R2 Standard (32-bit and 64-bit) with SP1 or later
  SQL Server 2008 Enterprise (32-bit and 64-bit) with SP2 or later
  SQL Server 2008 Standard (32-bit and 64-bit) with SP2 or


  VMM Library Server
  The VMM library server is essentially a store for the physical fi les that serve as the building blocks for Hyper-V hosts, virtual machines, and services. The templates and metadata are actually stored in the VMM database. Examples include the following:
·         VM and service templates
·         Service deployment confi gurations
·         Virtual hard disks
·         ISO images
·         Scripts
·         Several types of profi les
·         Stored virtual machines and services
·         Update baselines and catalogs


  The general requirements for the VMM library server include the following:
·         The VMM library server must be a member of an Active Directory domain. If that domain differs from the one containing the VMM server, a two-way trust relationship must exist between the two domains.
·         The name of the server cannot exceed 15 characters.
·         VMM includes support for highly available library shares on a failover cluster.

·         VMM requires an SMB/CIFS-based fi le server. It does not support case-sensitive fi le servers such as Windows Services for UNIX or NFS.

   Hardware component                           Recommended Hardware Requirement
   Processor                                               Dual x86 or x64 CPU, 3.2 GHz or faster
   RAM                                                       2 GB
   Disk                                                       Depends on the number and size of library files

   Software Requirements
   Software Requirement                                                     Notes
   A supported operating system               A core or full installation of Windows Server 2008 R2                                                                           (Standard, Enterprise, or Datacenter) with SP1 or later (x64). If                                                             the VMM library is clustered, only Enterprise or Datacenter SKU                                                            is supported.A core or full installation of Windows Server 2008                                                              with SP2 (x86 or x64). If the VMM library is clustered, only                                                                  Enterprise or Datacenter SKU is supported. Windows Server                                                                  2008 R2 with SP1 or later is the preferred operating system.
  Windows remote management
(WinRM) 1.1 or 2.0
                                                 WinRM 1.1 is included in Windows Server 2008 and the WS-                                                                 Management service is set to start automatically (delayed start).WinRM                                            2.0 is included in Windows Server 2008 R2, and theWS-Management service                                        is set to start automatically (delayed start).


Sunday, August 31, 2014

Cloud Computing and Big Data: Threat to IT Jobs and Over Hyped?

There are two disruptive technologies which have gained significant traction in the world economy: Cloud Computing and Big Data.

Lets discuss some opportunities and threats and realistic scenarios regarding these technologies.

Cloud Computing:

The world is seeing significant transformation in business with the arrival of cloud. In this blog, our focus is on public cloud.

More and More large and small enterprises are moving to public cloud hosted by Amazon, Microsoft and HP. Software as a Service (SAAS) is the hottest service available. All small and big software vendors have started hosting their software on the public and offer as a service. This frees up organizations headache in purchasing hardware and software.
Pay as you go makes lots of sense to an organization. There is faster turnover  to transition to service in cloud then in your own Data centers.

Since multiple vendors are hosting their software on cloud and offering it as a service, they are also taking ownership of availability of service and management of service. This affects IT jobs in Operation. As more and more servers are built on cloud, the less jobs remain for operations team. This results in lots of jobs to me lost. This reduces the cost of an enterprises. So the managed services model of the IT companies is dying. Dont be surprised if managed service model is completely dead in 5 years.

Now there is an interesting issue here. The challenge for an enterprises will be multiple vendor management. There will be a situation when there will too many software vendors to be managed so that  enterprise starts bringing the cloud back to its Data center. Its a possibility. Circle of life. Lets wait and watch


Big Data

Let me start with the statement that Big Data technologies are awesome and its potential for enterprise is enormous.
But when we talk about business opportunities in big data solution from IT perspective, i think its over hyped.

Let em explain you why.

Its an awesome opportunity if you have to stand up multi node clusters for big data. But this is not true anymore thanks to Amazon and Microsoft Azure. I dont need to set up expensive cluster to get big data solution for my enterprise. I can just go to Amazon EMR or Microsoft Azure HDinsight and spin up and instance and get my big data solution in few hundred dollars.

SO for IT companies Big Data wont make much money too.





Thursday, August 28, 2014

Amazon Web Services: AWS elasticity Understanding and using Bootstrapping

In this blog we will discuss AWS elasticity Understanding and using Bootstrapping

Bootstrapping
-The process of automatically setting up your instances
-Example
   -Install Windows Updates
   -Create Custom DNS names
   -Register with an ELB
   -Mount additional drives
   -start specific services
   -Copy files from S3
   -Install latest version of the software

   -Modify firewall configuration



Bootstrapping tools and tricks
-Cloud-init is installed on LINUX AMIs by default
-EC2 config (windows service) is installed on the windows
   -Windows activation
   -Hostname changes
   -Sysprep
   …..and more
-Scripting ( bash, powershell) or config management(chef, puppet)
-Dynamic Information http://169.254.169.254/latest/ 


Understanding User data
-Limited to 16K in sinze
-Can be set through AWS management Console or AWS command line tool
-Linux:embed scripts with # (cloudinit)
-Windows:embed scripts with <script>(EC2config)
-Also interact with AWS services using ec2 command line tools

Bootstrapping Best practices and Principles

-Creating a good Bootstrapping AMI is great...but theres more
-Dynamic as possible:Discovery or registration of instances
-Dyanmic as possible: graceful departures
-Example: MySQL replica failure(Died, poor performance....)
   -Failure Detection ( Ping, slow slave status.....)
   -smooth addition (Duplicate source DB instance  id, use as an api, maual)


Tuesday, August 26, 2014

Amazon Web Services: AWS Elasticity Principles of elasticity

Traditional/AWS Architecture
-Traditional architecture is based on peak load

-AWS architecture provisions resources on demand / as needed


Elasticity Principle #1: Automate
-        -  Manual intervention= failure to meet demand
-        -  Elastic scaling = Automation
-        -  Preconfigured AMI
-         - Bootstrapping principles

Elasticity Principle #2: Loose Coupling
-          -Tightly coupled applications- apps where one component specifically depends on another specific , single      unit (creates failure or poor performance)
-       -   Loose coupling design uses AWS services or horizontally scaled design. you will need to recode your application.




Elasticity Principle #3: Staying stateless
-Avoid statefull application that stores state information on a single instance
-Instead store state information on a speedy, redundant, shared location (ex. dyanmeDB)


Elasticity Principle #4: Horizontal scaling
-Large instances will eventually hit a limit ( cpu, memory etc)
-Horizontal scaling and design grows without bounds






Monday, August 25, 2014

Amazon Web Services: Global Best Practices

I will be writing series of blog on Amazon Cloud. This blog is about Global best practices for public cloud created by Amazon

Why should you use the AWS cloud?
-Service based vs Resource based
-Instant Delivery
-Non-Commit architecture
-predictable, pay as you go cost model
-proven, expert design


Best Practice #1: Redundant Design
-No Single point of failure
-Many AWS services assume redundancy

Best Practice #2: Loose coupling

-Loose coupling design principle : Each application component is independent
-Use Integration with AWS services instead of “Hard Ties”
Example
-Elastic Load balancer
-Simple notification service
-Simple Queue service 


Best Practice #3: Elasticity
-Grow or shrink architecture on demand
-Instances are “drones” reporting for duty: tasked during provision
-Bootstrapping/dynamic configuration
-health Monitoring\no faith in any of the system

Best Practice #4: Think Security
-AWS is secure…until they invited us! Shared Responsibility
-Use Iron wall security principles rather than fish net principles
-Encrypt everything you can
-Use MFA (multi factor authentication) when possible

Best Practice #5: Things work better in parallel
-New paradigm….IT resource limitations removed
-Farming: advantage of using one plow/five hrs or five plows /one hr?
-Use parallel processing( with elastic principles) to accomplish goals faster


Best Practice #6: Use Multiple storage types
-Multiple AWS offerings for multiple scenarios:
   -S3-Object Storage
   -Cloud front-Edge Caching (CDN)
   -EBS- block storage
   -RDS-Relational Database
   -Dynamo DB-NOSQL DB


Featured Post

Ichimoku cloud

Here how you read a ichimoku cloud 1) Blue Converse line: It measures short term trend. it also shows minor support or resistance. Its ve...