appagile-elasticsearch

Delploy Elasticsearch on AppAgile

Version: appagile-redis, redis Version 5.0.*

This deployment supports different storage models

- ephemeral storage standalone
- ephemeral storage cluster
- persistent storage with host mounted directories
  • An elasticsearch instance requires a lot of memory resources. It supports required/limited resources for memory and CPU consumptions.

If you want to create a cluster with hostmounted Volumes first setup a cluster with ephemeral storage. Then follow instructions to use host mounted volumes.

Prerequisites

Project

Create a project for elasticsearch cluster and its resources. We use in the following the project elasticsearch-prj.

$ oc new-project elasticsearch-prj

Templates

[cols=”2,10,1″,options=”header”]
|===

|Name |Description |Required

|appagile-elasticsearch-is-template
|The imagestream for the STI Builder Image of Elasticsearch.
|yes

|appagile-elasticsearch-route-template
|Creates a route in order to access the elasticsearch service from extern.
| optional

|appagile-elasticsearch-standalone-ephemeral-template
|Template to create a standalone and ephemeral Elasticsearch Instance
|

|appagile-elasticsearch-cluster-support-template
|Template to create resource for a cluster deployment of Elasticsearch.
|This is required when deploying a cluster.

|appagile-elasticsearch-cluster-ephemeral-template
|Template to deploy a node as part of a cluster.
|
|===

Add templates to be used later. These Templates can be found at

https://gitlabappadev.tsi-af.de/OSE3/appagile-elasticsearch/tree/master/ose-artefacts

Put them into namespace openshift, so that they can be accessed from every project.

$ oc create -f ose-artefacts/appagile-elasticsearch-is-template.json -n openshift
 $ oc create -f ose-artefacts/appagile-elasticsearch-standalone-ephemeral-template.json -n openshift
 $ oc create -f ose-artefacts/appagile-elasticsearch-cluster-ephemeral-template.json -n openshift
 $ oc create -f ose-artefacts/appagile-elasticsearch-cluster-support-template.json -n openshift
$ oc create -f ose-artefacts/appagile-elasticsearch-route-template.json -n openshift

 Imagestream

Create the Imagestream for the Elasticsearch STI Builder Image.

NOTE If this image was pushed into the openshift registry then the imagestream was created automatically, it then has not be to created. Create it for convenience in namespace openshift.

oc process appagile-elasticsearch-is-template -v DOCKER_IMAGE_REPOSITORY= | oc create -f - -n openshift
$ oc process appagile-elasticsearch-is-template -v DOCKER_IMAGE_REPOSITORY=vmapgmucrep01.appacd.tsi-af.de:5000/public/appagile-elasticsearch | oc create -f - -n openshift

Check if imagestream has been created successfully.

$ oc get is appagile-elasticsearch -o yaml

Authorisation

The Elasticsearch pod needs to discover a service to figure out the topology of the cluster. Therefore the pod will be started with a serviceaccount ‘elasticsearch’. This serviceaccount has to be granted the view privileges to query resources.

As owner of the project execute

$ oc policy add-role-to-user view system:serviceaccount:elasticsearch-prj:elasticsearch -n elasticsearch-prj

Check binding.


$ oc describe policyBindings :default -n elasticsearch-prj
 Name: :default
 Created: 20 minutes ago
 Labels: 
 Annotations: 
 Last Modified: 2016-12-20 12:52:47 +0100 CET
 Policy: 
 ....
 RoleBinding[view]:
 Role: view
 Users: 
 Groups: 
 ServiceAccounts: elasticsearch
 Subjects:

….

= Standalone elasticsearch node with ephemeral storage

 Deploying a standalone elasticsearch node with ephemeral storage.

oc process appagile-elasticsearch-standalone-ephemeral-template -v IMAGE_STREAM_NAMESPACE=elasticsearch-prj,IMAGE_STREAM_NAME=appagile-elasticsearch:latest,MEMORY_LIMIT=4Gi,CPU_LIMIT=2000m,ES_CLUSTER_NAME=es-cluster,NODE_ID=node-01 | oc create -f -

Create a route (optional)

oc process appagile-elasticsearch-route-template -v HOSTNAME_HTTP=elasticsearch.elasticsearch-prj.appad3.tsi-af.de | oc create -f -

Scale up

oc scale --replicas=1 dc/elasticsearch-node-01

Verify

If a route was created you can use the rest calls below to verify if the elastic instance is up and running.

$ curl -XGET 'http://elasticsearch.elasticsearch-prj.appad3.tsi-af.de/_cat/health?v&pretty'
$ curl -XGET 'http://elasticsearch.elasticsearch-prj.appad3.tsi-af.de/_cat/nodes?v&pretty'

Cluster setup

Deploying a cluster constists of two steps
1) deploy the support resources
2) create a deployment for each node

If you want to create a instance with persistent storage the change the deployment afterwards. See chapter ‘Using hostmounted storage’.

create supporting resoures

oc process openshift//appagile-elasticsearch-cluster-support-template -v IMAGE_STREAM_NAMESPACE=elasticsearch-prj,IMAGE_STREAM_NAME=appagile-elasticsearch:latest,SOURCE_REPOSITORY_URL=https://github.com/mmaier2/elastic_conf.git | oc create -f -

The following resources should be created:
 - serviceaccount "elasticsearch"
 - service "elasticsearch"
 - service "elasticsearch-discovery"
 - imagestream "elasticsearch"
 - buildconfig "elasticsearch"

A build should be started and a new elasticsearch image should be created and be pushed to the openshift registry.

You can check with the command

oc get is elasticsearch -o yaml

You should find a new image tagged latest.

Create an Elasticsearch node
You can create as many nodes as you want. Set the variable ELASTIC_NODES to the value of your choice.

Use the snippet below to create a cluster with 3 nodes.

ELASTIC_NODES=3;
 for i in $(seq 1 $ELASTIC_NODES);
 do
 echo "Setting up Node: node-$i."
 oc process openshift//appagile-elasticsearch-cluster-ephemeral-template -v MEMORY_LIMIT=4Gi,CPU_LIMIT=2000m,ES_CLUSTER_NAME=es-cluster,NODE_ID=node-$i,ES_MINIMUM_MASTER_NODES=2 | oc create -f -

done

 

 Scale up

The instances are not started by default. To start them, they have to be scaled up.

For each instance do:

oc scale --replicas=1 dc/elasticsearch-node-$i

For the sample above do

ELASTIC_NODES=3;
 for i in $(seq 1 $ELASTIC_NODES);
 do
 echo "Scaling up Node: node-$i."
 oc scale --replicas=1 dc/elasticsearch-node-$i

done

Create a route (optional)

oc process appagile-elasticsearch-route-template -v HOSTNAME_HTTP=elasticsearch.elasticsearch-prj.appad3.tsi-af.de | oc create -f -

== Set cluster wide setting

curl -XPUT http://elasticsearch.elasticsearch-cluster.appa11.tsi-af.de/_cluster/settings -d '{
 "persistent" : {
 "discovery.zen.minimum_master_nodes" : 2
 }
 }'
 *NOTE* with this REST Call you can change the qorum after adding or removing nodes.

 Verify Cluster State

If a route was created you can use the rest calls below to verify if the elastic instance is up and running.

$ curl -XGET 'http://elasticsearch.elasticsearch-prj.appad3.tsi-af.de/_cat/health?v&pretty'

epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
 1482402238 05:23:58 es-cluster green 3 3 0 0 0 0 0 0 - 100.0%

$ curl -XGET 'http://elasticsearch.elasticsearch-prj.appad3.tsi-af.de/_cat/nodes?v&pretty'

ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
 10.1.3.23 17 63 8 1.06 1.20 1.13 mdi - U7w28ks
 10.1.0.9 22 37 8 1.31 1.22 1.18 mdi * 1-d8Ghv
 10.1.0.11 17 37 7 1.31 1.22 1.18 mdi - wxEs1v7

$ curl -XGET 'http://elasticsearch.elasticsearch-prj.appad3.tsi-af.de/_cluster/state'
 {"cluster_name":"rb-es-cluster","version":5,"state_uuid":"Dbaun06SQTidpo57Fe1ahg","master_node":"AilwqisUTO-1EOgPjPa97A","blocks":{},"nodes":{"nhb9HUuJRhi9ELEJbION0A":{"name":"nhb9HUu","ephemeral_id":"zMoLny0pSFCR8ltWSK75NQ","transport_address":"10.1.4.14:9300","attributes":{}},"AilwqisUTO-1EOgPjPa97A":{"name":"Ailwqis","ephemeral_id":"Bl6JvV74S0WpGqD-iDLB_A","transport_address":"10.1.2.22:9300","attributes":{}},"SshNu5kfRMqnArd4gPd_eA":{"name":"SshNu5k","ephemeral_id":"8mrjdyYxRxeQbNgxF7d9Vw","transport_address":"10.1.0.21:9300","attributes":{}}},"metadata":{"cluster_uuid":"6lSVhMJZTryR2TyWoZzuSQ","templates":{},"indices":{},"index-graveyard":{"tombstones":[]}},"routing_table":{"indices":{}},"routing_nodes":{"unassigned":[],"nodes":{"SshNu5kfRMqnArd4gPd_eA":[],"AilwqisUTO-1EOgPjPa97A":[],"nhb9HUuJRhi9ELEJbION0A":[]}}}

Delete resources

oc delete bc,dc,services,routes,templates,is -l app=elasticsearch

Using hostmounted storage

to be done

[%hardbreaks]
[qanda]
What should I use?::
Use persistent storage :-).

Backlog