Vector database sharding
Multimodal search
JaguarDB quantization
JaguarDB Vector API
Vector databases in 2023
JaguarDB in Docker
Setup JaguarDB with tar package
Setup JaguarDB on multiple nodes
Vector index sharing
How zeromove works
Video introduction
|
|
Setup
There are two types of servers for operation: Jaguar server and HTTP server. The Jaguar server is required.
The HTTP server is optonal. The HTTP server is a gateway to serve HTTP clients which do not rely on platforms and languages.
If the HTTP server is not deployed, then clients must be launched from the Ubuntu 22.04 platform.
We first describe how to set up Jaguar server, then the HTTP server in the following sections.
There are three methods for setting up Jaguar server on your Linux system.
1) Docker Container Deployment: The first method involves deploying JaguarDB using Docker containers. This approach offers ease of management and scalability.
2) Host-Based Installation and Cluster Configuration: The second approach is to install JaguarDB individually on each host and then establish connectivity between these hosts by sharing a common cluster configuration file. This method provides more control over the installation process and allows for fine-tuned cluster management.
3) Public Key Authentication and Cluster-Wide Installation Script: The third method involves setting up public keys on all hosts within the cluster. Subsequently, you can run an installation script from one node within the cluster to deploy JaguarDB on all the nodes in the cluster. This approach streamlines the deployment process while maintaining security through key-based authentication. You can consider these options for selecting the most suitable approach for your specific requirements when setting up JaguarDB on your Linux system.
Method One: In Docker Containers
To deploy JaguarDB, you can use docker to quickly setup JaguarDB. Note that jaguardb_with_http has the http server.
docker pull jaguardb/jaguardb
or
docker pull jaguardb/jaguardb_with_http
docker run -d -p 8888:8888 --name jaguardb jaguardb/jaguardb
or
docker run -d -p 8888:8888 -p 8080:8080 --name jaguardb_with_http jaguardb/jaguardb_with_http
To launch the JaguarDB client terminal to interact with JaguarDB server:
docker exec -it jaguardb /home/jaguar/jaguar/bin/jag -apikey demouser
or
docker exec -it jaguardb_with_http /home/jaguar/jaguar/bin/jag -apikey demouser
Please note that package jaguardb_with_http contains the HTTP gateway while the jaguardb package does not. Package jaguardb_with_http opens up ports 8080 and 8888, while jaguardb opens port 8888 only. Port 8080 is for HTTP traffic, and port 8888 is for data exchange between the HTTP server and the database server or between the client and the database server. Note that the benefit of a HTTP server is that it provides a REST API to clients that run on any platforms such as MacOS, Windows, and Linux (not just on Ubuntu 22.04).
Method Two: Node Based Installation
In this approach, you have the flexibility to download and manually install the JaguarDB program (found in the jaguar-n.n.n.tar.gz package) on each host within your database cluster. Subsequently, you will need to configure the $JAGUAR_HOME/conf/cluster.conf file, which should be distributed to all the hosts in your system.To initiate the process, visit the download page at www.jaguardb.com/download.html and follow the provided link to obtain the tarball on your host. We will present the following example to illustrate the installation process using this method.
If you intend to configure JaguarDB on three hosts with the following IP addresses: 192.168.2.100, 192.168.2.101, and 192.168.2.102, you can easily achieve this by copying the tar file, jaguar-n.n.n.tar.gz, to each of these hosts. Here is the command to do so using the Secure Copy Protocol (SCP):
scp jaguar-n.n.n.tar.gz 192.168.2.100:~/
scp jaguar-n.n.n.tar.gz 192.168.2.101:~/
scp jaguar-n.n.n.tar.gz 192.168.2.102:~/
Next, you can proceed to install JaguarDB on each of these hosts using the following command:
ssh 192.168.2.100 "tar zxf jaguar-n.n.n.tar.gz; cd jaguar-n.n.n; ./install.sh"
ssh 192.168.2.101 "tar zxf jaguar-n.n.n.tar.gz; cd jaguar-n.n.n; ./install.sh"
ssh 192.168.2.102 "tar zxf jaguar-n.n.n.tar.gz; cd jaguar-n.n.n; ./install.sh"
The script install.sh provides the flexibility to specify an alternative installation directory using the "-d " option. This allows you to install JaguarDB to a location of your choice, rather than the default location, which is "$HOME/jaguar".For example:
ssh 192.168.2.100 "tar zxf jaguar-n.n.n.tar.gz; cd jaguar-n.n.n; ./install.sh -d /opt/jaguar"
ssh 192.168.2.101 "tar zxf jaguar-n.n.n.tar.gz; cd jaguar-n.n.n; ./install.sh -d /opt/jaguar"
ssh 192.168.2.102 "tar zxf jaguar-n.n.n.tar.gz; cd jaguar-n.n.n; ./install.sh -d /opt/jaguar"
Please ensure that the directory "/opt/jaguar" is owned by the user account executing the SSH command. No specific user account is necessary for the installation and operation of the Jaguar server software.
A cluster configuration file "cluster.conf" should be prepared and distributed to all the nodes within the cluster:
vim cluster.conf
192.168.2.100
192.168.2.101
192.168.2.102
scp cluster.conf 192.168.2.100:~/jaguar/conf/
scp cluster.conf 192.168.2.101:~/jaguar/conf/
scp cluster.conf 192.168.2.102:~/jaguar/conf/
If you have installed Jaguar in /opt/jaguar directory, you should replace "~/jaguar/conf/" with "/opt/jaguar/conf/"in the commands above.
The Jaguar server can be started using the jaguarstart command, with the "$JAGUAR_HOME" directory indicating the directory where Jaguar is installed, either$HOME/jaguar or /opt/jaguar as illustrated in the above example.
cd $JAGUAR_HOME/bin
./jaguarstart
Method Three: Cluster-Wide Installation
This method uses a downloaded tar ball and a script to install JaguarDB software on all hosts for your database cluster. To implement this, ensure that you employ identical account credentials across all hosts. This process will establish trusted public authorized keys for seamless SSH login, eliminating the need for repetitive password input.
Suppose you have downloaded jaguar-n.n.n.tar.gz to host 192.168.2.100, you can proceed with the followg steps to setup cluster-wide installation of JaguarDB:
# On each node, i.e., 192.168.2.100, 192.168.2.100, and 192.168.2.102
$ sudo useradd jaguar
$ sudo passwd jaguar
Please ensure that you use the same password on all nodes. Subsequently, from a designated node, such as node 192.168.2.100, you can execute the following steps:
su - jaguar
tar zxf jaguar-n.n.n.tar.gz
cd jaguar-n.n.n
At this point, you should create a file containing all the nodes within the cluster:
vim HOSTFILE
192.168.2.100
192.168.2.101
192.168.2.102
Following this, all that remains is to execute the following command:
./install_jaguar_database_on_all_hosts.sh -f HOSTFILE
In the installation process outlined above, the establishment of trusted public keys will occur for the user account "jaguar" across all nodes within the cluster. If you would like to install Jaguar to a different directory, you can execute the following commmand:
sudo mkdir -p /opt/jaguar
sudo chown jaguar.jaguar /opt/jaguar
./install_jaguar_database_on_all_hosts.sh -f HOSTFILE -d /opt/jaguar
The Jaguar server processes can be initiated, on all the nodes in the cluster automatically, with the jaguarstart script:
cd $JAGUAR_HOME/bin
./jaguarstart
Using this method, you can efficiently set up and manage Jaguar server processes on all nodes within the cluster from a single node.
HTTP Server Setup
In tandem with the JaguarDB server, the HTTP gateway provides a universal endpoint for clients to seamlessly interact with Jaguar servers. This gateway serves as a proficient proxy, facilitating the relay of commands from clients to the backend database servers. Consequently, clients are liberated from the constraints of relying on a specific Linux platform. To set up the Jaguar HTTP gateway on any Linux system, users can follow these straightforward steps:
- Acquire the file: wget http://www.jaguardb.com/download/fwww_3.4.1.tar.gz
- Execute the ./install.sh script in the extracted tar file to start the setup process
- Navigate to the $HOME/fwww/bin_dir directory and run ./start_all_servers.sh
It is important to note that the HTTP gateway server process should ideally run on the same host as one of the Jaguar servers, although it can also be deployed on separate hosts within a local area network for enhanced flexibility. The value of configuration parameter JAGUAR_SUPER_ADMIN_API_KEY in the fwww.conf file should be read from the configuration file $JAGUAR_HOME/conf/server.conf on one of a JaguarDB servers. When you open the file conf/server.conf, you will see SERVER_TOKEN and you can use its value for the JAGUAR_SUPER_ADMIN_API_KEY entry in the fwww.conf configuration file. Note that JaguarDB server should be installed prior to setting up the HTTP gateway server. If JaguarDB server is restarted, you should restart the http server.
Once the http gateway is setup, you point your browser at: http://:8080/login.html to login to your account and manage your data, where is the IP address of the server where the fwww http server is installed.
Development Setup
For software developers looking to harness the power of the Jaguar vector database for advanced AI applications, there are two convenient methods to get started. You can either download the Jaguar Software Development Kit (JSDK) as a tarball, named "jaguar-n.n.n.tar.gz", or employ the simplicity of Python packages. With the JSDK or the pip installation, you can quickly dive into programming and engage with Jaguar servers to build cutting-edge AI solutions. The three approaches are described below:
JSDK Download Visit www.jaguardb.com/download.html to download JSDK and find examples in the client directory
REST Client Run the command "pip install -U jaguardb-http-client" for client programming over REST protocol. This requires setting up jaguar http gateway server to accept commands from the clients and send them to the jaguar server. This package is pure Python and does not reply on any platform.
Socket Client Run the command "pip install -U jaguardb-socket-client" for client programming over TCP/IP socket. This packages relies on Ubuntu 22.04 platoform and bypasses the http gateway.
Please note that the JSDK is developed for Ubuntu 22.04 Linux distribution. More OS platforms will be supported in the future. Users are encouraged to visit github.com/fserv/jaguar-sdk for guides and examples.
|
