Troubleshooting OpenStack Authentication and Networking

This tutorial gives you a complete description and steps to Troubleshooting OpenStack Authentication and Troubleshooting OpenStack Networking

Troubleshooting OpenStack Authentication and Networking and Service Works

The OpenStack Authentication service (Keystone) is a complex service, as it has to deal with underpinning the authentication and authorization for the complete cloud environment. Common problems include misconfigured endpoints, incorrect parameters being stored, and general user authentication issues, which involve resetting passwords or providing further details to the end-user.

If you would like to become an OpenStack  Certified professional, then visit Mindmajix - A Global online training platform: " OpenStack Certification Training Course ". This course will help you to achieve excellence in this domain.

Getting started to OpenStack Authentication

Administrator access is required to troubleshoot Keystone, so we first, configure our environment, so that we can simply execute the relevant Keystone commands.

# Administrator Credentialsexport OS_USERNAME=adminexport OS_PASSWORD=openstack
export OS_AUTH_URL=https://172.16.0.200:5000/v2.0 export OS_TENANT_NAME=cookbook

How to achieve OpenStack Authentication service

Carry out the following steps when encountering the problems presented.

[Related Article: Learn OpenStack Network]

Misconfigured endpoints

The Keystone is the central service that directs authenticated users to the correct service, so it’s vital that the users be sent to the correct location. Symptoms include HTTP 500 error messages in various logs regarding the services that are being accessed and clients timing out trying to connect to network services that don’t exist. To verify your endpoints in each region, perform the following command:

keystone endpoint-list

We can drill down into specific service types with the following command. For example, to show admin URL for the compute service type in all regions:

keystone endpoint-get --service compute --endpoint_type adminURL

An alternative to listing the endpoints in this format is to list the catalog, which outputs the details in a more human-readable way:

keystone catalog

This provides a convenient way of seeing the endpoints configured.

Authentication issues

From time to time, users will have trouble authenticating against Keystone due to forgotten or expired details or unexpected failure within the authentication system. Being able to identify such issues will allow you to restore service or allow the user to continue using the environment.

The first place to look will be the relevant logs. This includes the /var/log/nova logs, the /var/log/glance logs (if related to images), as well as the /var/log/keystone logs.
Troubleshooting accounts might include missing accounts, so view the users on the system using the following command:

keystone user-list

[Related Article: OpenStack Advanced]

After displaying the user list to ensure an account exists for the user, we can get further information on a particular user by issuing, for example, the following command, after retrieving the user ID of a particular user:

keystone user-get 68ba544e500c40668435aa6201e557e4

This will display output similar to the following screenshot:

This allows us to verify that the user has a valid account in a particular tenant.

[Related Article: OpenStack Dashboard]

If a user’s password needs resetting, we can execute the following command after getting the user ID, to set a user’s password to (for example) OpenStack:

keystone user-password-update --pass openstack 68ba544e500c40668435aa6201e557e4

If it turns out to be a user which has been set to disable, we can simply re-enable the account with the following command:

keystone user-update --enabled true 68ba544e500c40668435aa6201e557e4

There could be times when the account is working, but problems exist on the client-side. Before looking at Keystone for the issue, ensure your environment is set up correctly for the user account you are working with, in other words, set the following environment variables (for example using a user called Kevin):

export OS_USERNAME=kevinjexport OS_PASSWORD=openstack 
export OS_AUTH_URL=https://172.16.0.200:5000/v2.0 
export OS_TENANT_NAME=cookbook

How OpenStack Authentication service works 

User authentication issues can be either client-side or server-side and when some basic troubleshooting has occurred on the client, we can use Keystone commands to find out why someone’s user journey has been interrupted. With this, we are able to view and update user details, set passwords, set them into the appropriate tenants, and disable or enable them, as required.

[Related Article: Object Storage in OpenStack]

Troubleshooting OpenStack Networking

Network troubleshooting can be hard. Network troubleshooting in a complex distributed system like OpenStack can be even harder
OpenStack Networking is now a complex service with the introduction of Neutron, as it now gives users the ability to define and create their own networking within their cloud environment. Common problems for an OpenStack administrator include misconfigured Neutron installations, routing problems and switch plugin problems. Problems for users include misunderstanding the capabilities of Neutron or limitations imposed by administrators.

Getting started to OpenStack Networking

We’ll be troubleshooting Neutron installations so administrator access is required to troubleshoot this service. Ensure you’re logged in as root on our controller, compute, and network hosts and configure our environment to enable us to run various commands.

[Related Article:  Network in OpenStack]

To log into our hosts that were created using Vagrant issue the following in separate shells:

vagrant ssh controller 
vagrant ssh compute
vagrant ssh network

In our Controller and Network host sessions, as root, issue the following:

# Administrator Credentialsexport OS_USERNAME=admin 
export OS_PASSWORD=openstack 
export OS_AUTH_URL=https://172.16.0.200:5000/v2.0 
export OS_TENANT_NAME=cookbook

How to achieve it

Carry out the following steps when encountering the problems presented.

Cloud-init reporting Connection Refused when accessing Metadata

In an instance’s console log (when you issue nova console-log INSTANCE_ID) you may see lines such as:

There are a number of possibilities for this, but the result will be the same and we will be unable to log into our cloud instance because the instance was unable to have its SSH key injected into it.

Check that you have configured our physical interfaces on our network and compute nodes for use with OVS. As part of the installation and configuration, ensure that you have run the following command:

ovs-vsctl add-port br-eth1 eth1

Where eth1 is our physical interface and br-eth1 is the bridge created on this interface.

Check that your instance can route to the 169.254.169.254 metadata host from the gateway of the instance and if not create a route to this network. When subnets are created and a gateway is specified, it is assumed that this gateway address can route to the 169.254.169.254 address. If it can’t, you will see errors described in the sections as we saw. To create a 169.254.169.254 route on the instance itself, create the subnet with the following options:

quantum subnet-create demoNet1 10.1.0.0/24
 --name snet1 --no-gateway
 --host_routes type=dict list=true destination=0.0.0.0/0,nexthop=10.1.0.1 --allocation-pool start=10.1.0.2,end=10.1.0.254

By specifying –no- gateway, Neutron will inject the 169.254.169.254 route into the instance so it shows up in the instance routing table, but to provide a default gateway, we specify a destination of 0.0.0.0/0 and if appropriate the next hop in the route to allow that instance to access elsewhere.