Managing SolrCloud Using solrctl

Apache Solr includes the ability to set up a cluster of Solr servers that combines fault tolerance and high availability called SolrCloud, these capabilities provide distributed indexing and search capabilities, supporting the following features:

  • Central configuration for the entire cluster
  • Automatic load balancing and fail-over for queries
  • ZooKeeper integration for cluster coordination and configuration.

SolrCloud is flexible distributed search and indexing, without a master node to allocate nodes, shards and replicas. Instead, Solr uses ZooKeeper to manage these locations, depending on configuration files and schemas. Documents can be sent to any server and ZooKeeper will figure it out.

The Cloudera Distribution of Hadoop comes with solrctl.sh script in the "bin" directory of the installation folder which can be used to manage common workflows such as creating a config directory, upload solr configs to zookeeper, creating a collection etc.

Print available commnds with solrctl:

solrctl

Create a Solr config template directory

solrctl instancedir --generate $HOME/collection1

Upload the configuration directory to zookeeper

solrctl instancedir --create $HOME/collection1

Create a collection with the above configuration

solrctl collection --create collection1

SolrCloud will check the configuration name with the same name as the collection name and link them to each other.

Reloads a collection

solrctl collection --reload collection1

List all uploaded configurations

solrctl instancedir --list

List all collections

solrctl collection --list

Setting Up Python and Django In Windows

While setting Python and Django  in Linux and MAC is fairly straight-forward, it could be a little challenging to set up Python in Windows machine. In this tutorial, I will guide you through the steps:

Install Anaconda Distribution of Python

Anaconda Distribution of python comes with several packages and package manager such as conda that makes speeds up up the initial set up process for python. At the time of this writing, Anaconda Python is the world's most popular distribution of Python.

Download it from here: https://www.anaconda.com/download/#windows and choose your version of Python to download.

After you hit the download button, it may ask to enter your email in which case you may choose to skip and download the installer.

After the download is complete, double click on the downloaded installer and it will start a wizard which will install the Anaconda Python.Basically, you can continue hitting next leaving all the default settings as is. I have taken a few screenshots of the wizard.

Installer Screenshots

After successful installation, you can see from the start menu the programs that was installed. As you can see from the figure below, the following programs were installed:

  • Anaconda Navigator
  • Anaconda Prompt
  • Jupyter Notebook
  • Spyder IDE (Integrated Development Environment)

You can learn more about them as you make progress with your Python learning, but in this tutorial we will be using Anaconda Prompt.

Install Django

Django is a popular web development framework for Python. Anaconda Python includes Django in its distribution which can be installed using its package manager called "conda". Open the Anaconda Prompt program from the windows start menu and run the command "conda install django" to install Django.

Type y at the prompt and it should install django which you can check by running the command "python -m django --version"

Creating a django project

To create a django project, run the command "django-admin startpproject mysite"

As you can see from above figure, that a folder named "mysite" was created in the Documents folder. Note: "ls" equivalent in windows is "dir"

Running the server

Go to the mysite directory and run "python manage.py runserver" which will start the server.

Using PyCharm IDE

Although Spyder IDE was installed with Anaconda Python, PyCharm is the most popular IDE for Python and I will show you how to set up your PyCharm to use Anaconda Python. Download and Install the community version of PyCharm. After successful installation, open PyCharm from the windows start menu. At the start, it will ask you whether to create a new project or open an existing project. When you open a new project, you can select the django's mysite project that we just created earlier from the Anaconda Prompt.

If you already were using some other Python in your PyCharm, you can change your project to use Anaconda Python from now on by going to the settings under File and changing the Python Interpreter to Anaconda Python. See the screenshot below:

Whenever you create a new project in PyCharm, you can tell it use Anaconda Python by selecting the right interpreter as shown in the picture below:

Hope that helps. Happy coding!