Getting started with PHP and MongoDB

 

This post and video delve into the world of schemaless, NoSQL databases. NoSQL databases are powerful and interesting ways of storing data at scale. Because they do not have a traditional table that dictates what kind of data needs to be written, NoSQL databases, like MongoDB, can accept data much more easily. There is also the benefit that, at scale, a cluster of multiple nodes can be hosting a common database and collection where as a traditional database cannot scale-out horizontally. 

In cloud computing, NoSQL databases are what drive the likes of Amazon, Google, Facebook and more. 

In order to learn more about them I wanted to try and replicate what I know on PHP and MySQL (a schema or relational database), and apply it to PHP with MongoDB. I did struggle a bit because of the way I was building the container and my lack of familiarity with the way Composer works. As such I thought I'd share my progress and provide the docker-compose file along with a run through of how to get it setup and running. 

The Dockerfile and Docker Compose script are located on my GitHub with no license and anyone can feel free to use the repository to quickly bring up and test a LAMP stack using MongoDB instead of MySQL or MariaDB. Just git clone, and run docker-compose up from the directory and it should build everything. 

Also included in the repo is a very simple HTML form that will write a Name and Age to the MongoDB instance. These two web pages are essentially just offered as a reference for how to connect and get started. 

Docker Compose source: https://github.com/JoeMrCoffee/MongoLAMP

Additional references that I found helpful:

Official documentation on using PHP with MongoDB:
https://docs.mongodb.com/drivers/php/

Official Docker image for MongoDB and Mongo-Express:
https://hub.docker.com/_/mongo
(This was the base for the docker compose minus the Apache PHP image)

Another great tutorial which is worth a look as well.
https://www.javatpoint.com/php-mongodb

Recover a broken NextCloud

This is not intended to be a definitive guide. I am just sharing my experience after crashing my NextCloud container. 

I performed an upgrade on my "NAS" (Ubuntu server on an old laptop) by adding a second, larger external USB drive. This is not best practice, but works for me as a file backup. 

In adding the drive, I was trying to be careful, but obviously I wasn't careful enough. I changed the fstab file (this is where Linux knows what drives and filesystems to mount), to include the new drive. That generally worked fine, but I was mounting using the /dev/sda~/dev/sdb terminology for the drives. When Linux boots it checks the storage driver for what hardware and assigns it to the /dev/ based on its type (nvme will show up /dev/nvme####, SATA drives will usually show up as /dev/sdX#). What happened was because Linux assigns the /dev type at boot, when I rebooted with the new external drive the /dev assignments changed. SURPRISE!!! That then meant 2 out of 3 of my fstab assignments are wrong. They tried to mount backwards.

So that wasn't good. My root drive was using the UUID, which is better and should be used. For the other 2 external drives I changed to using the drive label that can be set when initially formatting the drives. Something like the below:

LABEL=usbbackup /media ntfs defaults 0 0

This was fine for getting everything straightened out. However, in the process of the drives mounting in the wrong order, my NextCloud and MariaDB containers got out of whack. I couldn't log in and just had an "Internal server error" message. 

The help pages mentioned that one can use maintenance mode and using the occ tools to help fix something similar. Unfortunately, the Docker install doesn't have a lot of tools, like 'sudo' to help test. Fortunately, all the data for both containers was 100% still intact as it was mounted using the volume parameter from the docker-compose file. This means, in theory, you can just nuke the containers, rebuild new containers with the same volume points, and get back in business. 

At this point, I want to stop and say that while this is possible and worked for me, be very very very careful that the data is still intact for both the database and NextCloud instances. If anything beyond the defaults in the docker-compose file was changed - i.e. additional packages or other things were added in the container - the next steps will wipe all of that. 

Steps I used:

Stop the containers:

  sudo docker stop nxtcloud nxtclouddb 

Note: Container names are defined as such in my version of the docker-compose file. Running sudo docker ps -a will show the current names of all the containers.

Backup the container data directories to somewhere safe. Leave the data intact in the original location. 

Note: If you can't get the containers restarted and need to truly nuke and pave, all the files in NextCloud are stored under the NextCloud data directory. On the container this would be /var/www/html/data/, on the local volume it would be under the <NextCloud install directory>/data/data/ on the host system using my docker-compose file.

Delete the containers:
  sudo docker rm nxtcloud nxtclouddb

Download the latest version of the NextCloud image:
 sudo docker image pull nextcloud:latest

Update the docker-compose file to specify the MariaDB version as 10.5:
 image: mariadb:10.5

This step is run because on my first go, NextCloud had issues with the latest version of MariaDB (version 10.6 I believe). The docker-compose pulled the latest image version by default.

Recreate and run the containers. From the directory of the docker-compose and data directories run:
 sudo docker-compose up -d

Once the container names appear, verify they are running correctly:
 sudo docker ps

Now open to the NextCloud browser page. In my case it asked to update the NextCloud instance. This triggered everything to get sorted out in NextCloud, and I could log in and everything was fine. Users can log in, data was intact, and things were copacetic.

The updated version of my NextCloud docker-compose file to reflect the change to use MariaDB version 10.5 is HERE.

Moral of the story, always stop running services on a server, especially the Docker containers BEFORE making a hardware update to the system. No matter how simple it may seem, put the host system in maintenance mode first.

More information on /etc/fstab configs is available here:
  https://linuxhint.com/write-edit-etc-fstab/
  https://help.ubuntu.com/community/Fstab

Hopefully this helps demonstrate how volumes can be used to keep container data persistent, along with the flexibility of working with containers. I was lucky that the upgrade was able to trigger a re-sync of the NextCloud instance and the database. Some more information about doing this manually is in the NextCloud Doc page.
  https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/occ_command.html

Practical Programming using Python with Files

 

This tutorial demonstrates how to use Python to open and read from files in a local computer. Python is a powerful language, and the relative ease it has in opening and reading from text files makes it ideal for doing simple things (like calculating an average as showed), to more advanced (say updating 1000s of files in a single batch, or comparing different output from a research survey). There is an endless number of useful things Python can help automate, and it doesn't take a full time developer or software engineer to use.

The source code that was created is below. 

AverageNumberOfLines

For those interested as well in how to write to files, the script I used to create the 100 files with random ends is also provided. This script works by reading a sample file (Lorem Ipsum text in the example) and then randomly chops between 10 and 100 lines of text for 100 different test files. 

CreatingTextFiles

I hope it can be helpful for those interested in programming and eager to see some simple, but useful, use cases.

Creating Animations in GIMP

 

This article and video is a bit more artistic / creative compared to more of the recent ones. Most recently we've been exploring server-side services and writing scripts. This article looks to tackle creating GIF animations using GIMP (GNU Image Manipulation Program).

The video above plays pretty quick because I just created a very simple scene for demo purposes. The possibilities are really infinite. You could use those same steps to make a banner add for your company, or create a web animation that plays on load of a website. It could also be used for doing some animations with Apps like on iOS or Android (though GIFs in Android need a webview to run). Even something as simple as just quick UI or text tutorials can be done - that's how I made the simple animations in my BASH script blogs. 

Hope this can be helpful for some of you out there. I think GIMP is a wonderful tool - unfortunate name, but wonderful tool - that really can do some outstanding photo and image manipulation. It's also open source and free to use on Windows, Linux and MacOS so if you learn it once you can use it anywhere.

Final image from the tutorial - playing around with the number of frames and delay between frames (currently 180 ms) can create a smoother playback.

More BASH - Auto Sorting Files

BASH SMASH REHASH

I'm sharing a recent script I wrote that might be helpful. 

I had a picture folder on my NAS that was just simply way too large for use on a relatively slow Wifi connection. Essentially hundreds of picture and video files from our family photos.

What would happen on our computers is that when you want to open one file it takes forever as the file viewer tries to cache all the images in the folder to make flipping through them faster. That's a nice feature, but not when the folder / directory is dozens to hundreds of gigabytes in size. 

This script essentially prompts for which folder to sort (must be absolute), then prompts for how many files the user wants to sort per folder. The example below prompts to sort a directory with around 200 files into directories of 20 files apiece. 

Link to the code: autosort.sh

I included a lot of comments so hopefully the script makes sense. Again just a useful tool and might be helpful for those interested in shell scripting.