MediaWiki Extension CirrusSearch and Elasticsearch Setup

De­vOps and SRE

Lin­ux Serv­er

This is a short man­u­al how to set-up Elas­tic­search to be used with the MediaWiki's ex­ten­sion Cir­rusSearch. You should choice an ap­pro­pri­ate Elas­tic­search ver­sion de­pend­ing on your Me­di­aWi­ki ver­sion. Cur­rent­ly I'm us­ing Me­di­aWi­ki 1.38 and it is rec­om­mend­ed to use Elas­tic­search 6.8.23+ with it. This ver­sion runs well over open­jdk-11 which is the de­fault Ja­va ver­sion on Ubun­tu Serv­er 22.04.

Set­up Ja­va and Javac

To check and switch the cur­rent ver­sion of Ja­va and Javac you can use the fol­low­ing com­mands.

sudo update-alternatives --config java

There are 2 choices for the alternative java (providing /usr/bin/java).

  Selection    Path                                            Priority   Status
------------------------------------------------------------
  0            /usr/lib/jvm/java-11-openjdk-amd64/bin/java      1111      auto mode
* 1            /usr/lib/jvm/java-11-openjdk-amd64/bin/java      1111      manual mode
  2            /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java   1081      manual mode

Press <enter> to keep the current choice[*], or type selection number: 1

sudo update-alternatives --config javac

There are 2 choices for the alternative javac (providing /usr/bin/javac).

  Selection    Path                                          Priority   Status
------------------------------------------------------------
  0            /usr/lib/jvm/java-11-openjdk-amd64/bin/javac   1111      auto mode
* 1            /usr/lib/jvm/java-11-openjdk-amd64/bin/javac   1111      manual mode
  2            /usr/lib/jvm/java-8-openjdk-amd64/bin/javac    1081      manual mode

Press <enter> to keep the current choice[*], or type selection number: 1

If you are us­ing Elas­tic­Search 5.x it re­quires

sudo apt install openjdk-8-jre-headless 
sudo apt install openjdk-8-jdk-headless
sudo update-alternatives --config java
sudo update-alternatives --config javac
sudo systemctl restart elasticsearch.service 
curl 'http://127.0.0.1:9200' # do a test

Elas­tic­Search

Ela­sic­Search 6.5.4, кой­то е ин­с­та­ли­ран ра­бо­ти с open­jdk-11, ко­я­то е вер­си­я­та по под­раз­би­ра­не в Ubun­tu 20.04. При усло­вие, че въз­ник­нат проб­ле­ми със ста­бил­ност­та, ще се на­ло­жи пре­ми­на­ва­не към Ela­sic­Search 5.6.16, ко­я­то ра­бо­ти с openjdk‑8. За под­роб­нос­ти: виж тук.

sudo apt install -y apt-transport-https default-jdk default-jre

cd ~/Downloads
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.16.deb
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.5.4.deb
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.8.23.deb
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.0.1-amd64.deb

sudo apt install ./elasticsearch-6.5.4.deb

# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.0.1-amd64.deb

sudo systemctl enable elasticsearch.service
sudo systemctl start elasticsearch.service
sudo systemctl status elasticsearch.service

На­ма­ля­ва­не на раз­ре­ше­на­та па­мет. До­ри и та­зи ми се виж­да мно­го в bg​.trivictoria​.org e 128m. Ако има мно­го ед­нов­ре­мен­ни за­яв­ки мо­же да над­хвър­ли на­лич­на­та па­мет и да се счу­пи. От дру­га сра­на ако е мно­го мал­ко пак се чу­пи.

sudo nano /etc/elasticsearch/jvm.options

#-Xms1g
#-Xmx1g
-Xms512m
-Xmx512m

До­ба­вя­не на ди­рек­ти­ви за ав­то­ма­тич­но рес­тар­ти­ра­не в system.d unit‑а.

sudo cp /lib/systemd/system/elasticsearch.service ~/Downloads/elasticsearch.service.default
sudo nano /lib/systemd/system/elasticsearch.service

[Service]
# В края на секцията
Restart=always
RestartSec=3

При­ла­га­не на про­ме­ни­те.

sudo systemctl daemon-reload
sudo systemctl restart elasticsearch.service
sudo systemctl status elasticsearch.service

Про­вер­ка.

curl 'http://127.0.0.1:9200'

{
  "name" : "HFrziWt",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "5qMVv8CHT3q2vv1sd8hLOw",
  "version" : {
    "number" : "6.5.4",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "d2ef93d",
    "build_date" : "2018-12-17T21:17:40.758843Z",
    "build_snapshot" : false,
    "lucene_version" : "7.5.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

За да за­поч­не ре­гу­ляр­но ин­дек­си­ра­не на съ­дър­жа­ни­е­то на уики­то, спря­мо кон­фи­гу­ра­ци­я­та, напра­ве­на в /var/­www/­*/­Local­Sett­ings.php и до­ку­мен­та­ци­я­та на mw:Extension­:­CirrusSearch тряб­ва да напра­ви пър­во­на­чал­на ин­дек­са­ция, да се из­пъл­нят за­да­чи­те, ко­и­то ще съз­да­де тя, да се ре­ге­не­ри­ра ин­дек­са на съ­дър­жа­ни­е­то и от­но­во да се из­праз­ни опаш­ка­та със за­да­чи­те. За цел­та мо­гат да се из­пол­з­ват скрип­то­ве­те за под­дръж­ка, опи­са­ни в сек­ци­я­та Me­di­aWi­ki.

mw-maintenance-elasticsearch-index.sh
mw-maintenance-runJobs.sh cli
mw-maintenance-rebuildAll.sh
mw-maintenance-runJobs.sh cli

В до­пъл­не­ние е раз­ра­бо­тен скрип­та elasticsearch​-watch​.sh, ка­то чрез crontab за­да­ча се пра­ви пе­ри­о­дич­на про­вер­ка и при не­об­хо­ди­мост рес­тар­ти­ра­не. Скрип­та из­пра­ща пис­мо до vectoria@altclavis.com, ако настъ­пи съ­би­тие.

sudo crontab -e

# ElasticSearch Watch
*/5 * * * * /usr/local/bin/elasticsearch-watch.sh

Ref­er­ences

• Bit­Launch: How to in­stall Elas­tic­search on Ubun­tu 20.04 LTS
• Com­put­ing for Geeks: In­stall Elas­tic­search 6.x on Ubun­tu 18.04 LTS
• Me­dia Wi­ki: Extension:CirrusSearch
• Phabri­ca­tor: Extension:CirrusSearch
• Me­dia Wi­ki: Cir­rusSearch Talk – Ja­va ver­sion com­pat­i­bil­i­ty
• Mincong's blog: GC in Elas­tic­search – Ba­sic in­for­ma­tion about garbage col­lec­tion (GC) in Elas­tic­search, JVM op­tions, GC log­ging
• Foojay​.io: Han­dling JDK & GC Op­tions Dy­nam­i­cal­ly in Elas­tic­search
• Elas­tic­search Doc­u­men­ta­tion: Im­por­tant Elas­tic­search con­fig­u­ra­tion
• Elas­tic­search Doc­u­men­ta­tion: GC log­ging

Ac­cess Elas­tic­search via SSH

• SSH Per­sis­tent Tun­nel and SSHFS Mount via "sys­temd" units

---

-RE­VIEW-

Elas­tic­Search

Re­quired by MW:Extension:CirrusSearch, some oth­er mw:extensions and some ex­ten­sions of NextCloud.

~/Downloads/elastic-search/
sido apt install ./elasticsearch-5.6.16.deb

До­ба­вя­не на фун­к­ция за ав­то­ма­ти­чен рес­тарт за elasticsearch.service, тъй ка­то по ня­кой път се „чу­пи“:

sudo nano /lib/systemd/system/elasticsearch.service

    [Service]
    ...
    Restart=always
    RestartSec=3

Ja­va

Down­grade re­quired by MW:Extension:CirrusSearch and Elas­tic­Search 5.6.16.

Reg­is­ter: De­vOps and SRE

Идентифициране

Разработване

Авторизиране

Утвърждаване

Spas

Spas

5.07.2022

30.08.2022

27720

30792