What are we doing here?

This blog includes a series of videos and references to help new users or enthusiasts better understand how to use open source and free technology tools. The quick links includes more information for accessing many of the tools covered along with other references to learn more for taking advantage of these tools.

Click HERE to see the full list of topics covered!

Adding Comments to Posts - MongoDB and PHP

This article explores adding a comments section to blog or forum posts. 

Using my previous project 'OutNoteOrganizer' - which still needs a better name, but all the good ones I could think of were taken - I thought about the design and felt it needed to have a comment section. 

Similar to say for example Reddit, though not public, teams or groups of people could want to leave comments on the initial post. The other thing I didn't like about the initial design was that anyone in the associate group could edit and change the content without a good way from the UI to track what was changed. 

Now the current version only allows the original post author to edit the post, and other group members with access to the post can leave comments.

How I got here?

In terms of how I visualized the code structure, previously I was hung up thinking that comments should be fields in the post itself. For example comment 1, comment 2, etc. However, while it might be possible to do that, it's complex if you want to have an unlimited number of comments to a post. Adjust the concept a bit and you can think of comments that link to the post but are stored in a different table (in MongoDB a different collection) called 'comments'.


Again with MongoDB, there is no need to 'prime' or 'setup' the table. Just from the code tell the DB what fields you want and it will accept the values as a BSON object. 

Once you have an HTML form filled in you really only need this code to create a new collection called 'comments'. 

$col = $db -> comments; //Point the query at the new collection comments (this will create the collection)

//Collect data from an HTML form

$username = $_POST['author'];

$comment = $_POST['commentcontent'];

$postname = $_POST['postname'];

//Create a PHP array to insert into MongoDB

$insertcomment = ['author' => $username, 'comment' => $comment, 'postname' => $postname];

//Make the insert query / connection - the if / else just catches an error

if ($postcomment = $col->insertOne($insertcomment)){

        echo "<h3>Comment added to $postname. </h3>";

}

else { echo "<h3>Error occured while adding that comment. Please contact your IT administrator.</h3>"; }

I also wanted to create a hidden form so that it only appears when a user clicks 'ADD COMMENT'. Initially I was thinking using Javascript and passing an innerHTML call would work. It does for the action, but because the comments need to collect some hidden data (i.e. the post title, the logged in user, etc) in PHP variables, the hidden variables need to exist in the PHP code before the Javascript action takes place. 

Explanation: PHP is a server side language meaning all the actions are requests to the server on page load. If the page loaded, but the PHP variables are not present during page load, the script won't know about them. Javascript is a run-time in the web browser itself. This makes it good for these on page actions - like <onclick='dothisfunction()'>. However, passing additional HTML after the page loaded will omit the variables we need.

It turns out that Javascript has styling properties as well. What they do is allow for CSS changes to be applied to specific elements identified by the 'id' tag.  What we can do is rather than pass new HTML code to the page - which causes issues with the PHP variables needed - we can more simply change the fields from 'hidden' to 'visible'. This way the PHP variables are known to the script, but the same visual result of handling a pop up can be achieved. 

In code - HTML with PHP variables - called using echo in PHP

<button onclick='addcomment()'>ADD COMMENT</button><br><br>

<div id='AddComment' style='visibility: hidden;'>

<form method='post' action='submitcomment.php' id='insertComment'>

<textarea class='giantinput' name='commentcontent' placeholder='Add a comment'></textarea>

<input type='hidden' name='author' value='$loginuser'>

<input type='hidden' name='postname' value='$postname'>

<br><br><input type='submit' form='insertComment' value='Submit'></form>

</div>

In code - JS

function addcomment() {
    var commentform = document.getElementById('AddComment');
    
     if (commentform.style.visibility === 'hidden') {
        commentform.style.visibility = 'visible';
     }
    
     else { commentform.style.visibility = 'hidden'; }
 }

The above is fairly basic as an example implementation. For better accuracy, it may be the case that the HTML form for the comments needs to collect more than just the post title or name, but also say the post author, the date, or better the object ID to ensure if there are say multiple posts with the same name all the comments do not get pulled into each post. 

Just wanted to share a bit of code and discovery using PHP and Javascript hand-in-hand to create a useful layout. Hope it helps. 

Full source code and Docker Compose to install is located on GitHub.

 

Intro to Minio S3 Object Storage

 


Minio  S3 object storage is powerful and easily run in a Docker, LXD, Podman or other container environment. Super easy to setup, super easy to understand, and the latest console has a ton of features to improve the intuitiveness of the platform. 

The command run in the video is the same as in the previous blog:

sudo docker run -it --name=miniotest -p 9000:9000 -p 9001:9001 minio/minio server /data --console-address ":9001"

To map a volume to the container one can create a directory on the local machine and add the -v <localvolume>:/data to the above command. That should help keep the buckets and related files available if the container is ever removed.

Example below from the container created in this test.



More information is below:

Happy to field any questions, just let me know. 



Integrate Firefox .tar into Linux native

Ubuntu 22.04 is here and with it are more polish and features. One feature / change is that Firefox is now a snap package. 

Snaps? What? 

Snaps are the way a lot of third party software gets bundled and packaged in Ubuntu. Developed by Canonical, they are a good way to run applications in a sandboxed environment on the PC. Many, many applications that are either not focused on the Linux community, or otherwise not devoting resources to building applications to run natively on Ubuntu, Fedora, Red Hat, Debian, Arch, etc. can be installed and run on the Linux Desktop by way of Snaps. Full list here: https://snapcraft.io/store

The major competitor to Snaps is Flatpak which is another great project. I may do a blog or video comparing the two in the future, but essentially both help package hundreds of applications for Linux that run in an isolated environment on the host system. Unlike Docker or Podman, these contained applications function pretty much like a standard desktop application with icons, launchers, etc. 

Now Firefox is a member of the Snap family. Great. Why the blog post?

Well Snaps have a reputation of occasionally being somewhat slow to load. Once running fine, but that initial click seems to take a while even with an SSD. I first learned about this with Ubuntu 18.04 running an admittedly 'netbook-like' device (low power laptop) when clicking the most basic calculator seemed to take several seconds. It wasn't until I learned about Snap packages vs native apt or .deb packages that I found the issue. Running calculator installed using the command 'apt install gnome-calculator' just pops right up, the Snap version takes a few seconds longer than it should. 

This is true with the Firefox Snap as well. Being honest, I was kinda looking for it, because when I upgraded my laptop from Ubuntu Mate 21.10 to Ubuntu Mate 22.04 one of the things highlighted was that Firefox would be removed and replaced with a Snap. Indeed, when first using it I had lost all my settings (not too many), and it seemed slower to boot. No apt package available, I set out to install using the Firefox download.

If you download Firefox it gives you a Zip file - technically a .tar.bz2 zip file - that you can unzip and just run Firefox from the 'firefox' directory - just double-click the file firefox-bin. To make this more integrated in the desktop experience do the following. 

Download and unzip Firefox from the official Mozilla website: https://www.mozilla.org/en-US/firefox/new/?redirect_source=firefox-com

Based on the location of the files create a firefox.desktop file under /usr/share/applications. *This will require 'sudo' or 'root' permissions.

Make the file:

  • Open a text editor - Text Editor, Pluma, Kwrite, Nano, VI, VIM - any will do.
  • Enter the following:
    [Desktop Entry]
    Version=1.0
    Name=Firefox
    GenericName=Internet Browser
    Type=Application
    Categories=Network;
    Exec=/<extract location>/firefox/firefox-bin
    Icon=/
    <extract location>/firefox/browser/chrome/icons/default/default32.png
  • Save as firefox.desktop in the /usr/share/applications directory on Ubuntu
  • Log out and log back in to confirm. 

That's it. 

Firefox will get displayed as a searchable application. In my case it even reappeared as the top of my favorites just like it had been saved before I upgraded.

Snaps are great for ease of use and running proprietary applications, such as games, or perhaps the odd communication platform. However, I think for something as commonly used as a browser, having something running on the operating system leads to a speedier experience. 

Hope it helps!

 

 

Fixing filesystem permissions in Flatpak

flatpak logo 

I had a problem with my Flatpak installation of Slack - I couldn't save files. 

I don't recall this being an issue before, but in my new computer with Ubuntu Mate 21.10, it was. 

Turns out the solution is fairly straightforward. 

Flatpak has the option to specify a file path which can then be written to. If this file path is then the default download folder everything is copacetic.

Steps:

Close Slack and make sure it is not running in the background. 

Run flatpak ps to ensure com.slack.Slack is not listed.

Create a directory for storing the download files. This can be in your file manager (Nautilus, Caja, Dolphin, Thunar, etc.), or with the mkdir command.

Once created you can run the following:

flatpak override com.slack.Slack --filesystem=<your full directory path>

*This command needs to be run as root in my testing (use sudo). 

Then start the Slack application again and go to Preferences -> Advanced -> Download location. Set it the same as the directory path just mapped. 


That's it. Files should now be able to be saved to that specific directory. 

Hope it helps anyone who needs to do something similar in Slack or another Flatpak application.


Nextcloud External Storage and Apps

 

This post and video go through how to add external storage in Nextcloud and introduces the wide number of applications that can be used to tailor the functionality of Nextcloud. 

For the external storage, the example is S3 object storage from a Minio container. Minio is a fantastic project that allows for locally hosted S3 API equipped storage. It's also nice because it is quite easy to start and get running, particularly on Docker. 

The command I used to make the Minio test container is below:
sudo docker run -it --name minios3 -p 9000:9000 -p 9001:9001 minio/minio server /data

*Update: Minio's latest image (tested 4/2022) needs to define the console port or it tries to auto find a port that was a bit hit or miss for me.
Revised:

sudo docker run -it --name=miniotest -p 9000:9000 -p 9001:9001 minio/minio server /data --console-address ":9001"

I did need to make some cuts in the video which is why the mouse jumps in a couple of places. Most notably, I initially had the wrong IP address of the Minio instance. This was essentially because in the test environment both containers were running on the same system and couldn't connect using the host IP. External systems wouldn't have had that issue. Around minute 7:25 the change will show moving from the host IP to the actual IP of the container. 

In more detail:

Docker, Podman, and other container management tools assign IP addresses to each container service. When running as a group, say if using a pod and Kubernetes, or running the containers together with Docker Compose, the containers are part of a single network and can identify each other by the service name. 

In this example, the containers were created separately, and being on the same host where unable to reach each other using the address I provided. Changing to the Minio specific IP was all that was needed, but as this was out of scope for the video - and honestly not something that would normally come up, so I chose to omit the debugging.

Some more information and resources about both Nextcloud and Minio are below.

Nextcloud Docker

Minio

Minio Quickstart

Minio Docker

Creating a web application with a MongoDB backend and Docker


This is a bit of a POC that turned into a project.

I was playing with MongoDB to create a front-end that would store data in a user friendly way. I didn't want to take on things like WordPress or others, and had an idea for a note taking application that could be on-prem, and put teams or catalogs first with grouping.

I think there is quite a bit more that could be done, like adding reminders/due dates to the posts/notes and adding more management to groups so new authors can add more people to a project/group. For now this is being used to demonstrate the capabilities of PHP with MongoDB.

A killer feature in MongoDB is the ability for it to not need to be primed in anyway. This feature allows for this app install using Docker to exist. In a traditional/relational database a user needs to setup the database and table structure in order for data to be inserted. With NoSQL/MongoDB you can just point an app at the DB and with the correct credentials start adding and editing a whole DB in the service itself. This is what I do in the initial login screen. I check if the DB and admin exist, and if not, they get automatically populated so a user on a completely fresh install can just start using the application. 

Currently this application is far from perfect, but I hope it helps users get an idea of what an be done using PHP, HTML, CSS, and MongoDB together. I also hope it can be useful, and if anyone requires additional assistance, please feel free to leave comments on the GitHub page. I am not at the caliber of a professional open source developer, and have limited time, but I am interested in the feedback, and willing to assist as I can to further promote usage of the project.

I hope this can be a useful project either for internal team collaboration or simply as reference code for other users. The code is published with the permissible Apache 2.0 license (same as Apache and MongoDB that it builds off).

Notes:

Source code is here: https://github.com/JoeMrCoffee/OurNoteOrganizer

A word on TinyMCE

TinyMCE is an external text editor which is used to help enhance the ease for writing and editing posts. It is an external tool that requires connectivity to external networks in order to function. For environments that do not have external Internet connectivity a standard 'textarea' without rich formatting is employed.

As this project is made to be hosted on any on-prem environment, there is no TinyMCE API key provided or registered. Users can create their own keys based on their domains and needs. To remove the notification about getting started, users can follow the quick steps, create their own API key, and add it to line 8 of the newpost.php file.

More information on TinyMCE and getting started:

https://www.tiny.cloud/

https://www.tiny.cloud/docs/quick-start/


Getting started with PHP and MongoDB

 

This post and video delve into the world of schemaless, NoSQL databases. NoSQL databases are powerful and interesting ways of storing data at scale. Because they do not have a traditional table that dictates what kind of data needs to be written, NoSQL databases, like MongoDB, can accept data much more easily. There is also the benefit that, at scale, a cluster of multiple nodes can be hosting a common database and collection where as a traditional database cannot scale-out horizontally. 

In cloud computing, NoSQL databases are what drive the likes of Amazon, Google, Facebook and more. 

In order to learn more about them I wanted to try and replicate what I know on PHP and MySQL (a schema or relational database), and apply it to PHP with MongoDB. I did struggle a bit because of the way I was building the container and my lack of familiarity with the way Composer works. As such I thought I'd share my progress and provide the docker-compose file along with a run through of how to get it setup and running. 

The Dockerfile and Docker Compose script are located on my GitHub with no license and anyone can feel free to use the repository to quickly bring up and test a LAMP stack using MongoDB instead of MySQL or MariaDB. Just git clone, and run docker-compose up from the directory and it should build everything. 

Also included in the repo is a very simple HTML form that will write a Name and Age to the MongoDB instance. These two web pages are essentially just offered as a reference for how to connect and get started. 

Docker Compose source: https://github.com/JoeMrCoffee/MongoLAMP

Additional references that I found helpful:

Official documentation on using PHP with MongoDB:
https://docs.mongodb.com/drivers/php/

Official Docker image for MongoDB and Mongo-Express:
https://hub.docker.com/_/mongo
(This was the base for the docker compose minus the Apache PHP image)

Another great tutorial which is worth a look as well.
https://www.javatpoint.com/php-mongodb

Recover a broken NextCloud

This is not intended to be a definitive guide. I am just sharing my experience after crashing my NextCloud container. 

I performed an upgrade on my "NAS" (Ubuntu server on an old laptop) by adding a second, larger external USB drive. This is not best practice, but works for me as a file backup. 

In adding the drive, I was trying to be careful, but obviously I wasn't careful enough. I changed the fstab file (this is where Linux knows what drives and filesystems to mount), to include the new drive. That generally worked fine, but I was mounting using the /dev/sda~/dev/sdb terminology for the drives. When Linux boots it checks the storage driver for what hardware and assigns it to the /dev/ based on its type (nvme will show up /dev/nvme####, SATA drives will usually show up as /dev/sdX#). What happened was because Linux assigns the /dev type at boot, when I rebooted with the new external drive the /dev assignments changed. SURPRISE!!! That then meant 2 out of 3 of my fstab assignments are wrong. They tried to mount backwards.

So that wasn't good. My root drive was using the UUID, which is better and should be used. For the other 2 external drives I changed to using the drive label that can be set when initially formatting the drives. Something like the below:

LABEL=usbbackup /media ntfs defaults 0 0

This was fine for getting everything straightened out. However, in the process of the drives mounting in the wrong order, my NextCloud and MariaDB containers got out of whack. I couldn't log in and just had an "Internal server error" message. 

The help pages mentioned that one can use maintenance mode and using the occ tools to help fix something similar. Unfortunately, the Docker install doesn't have a lot of tools, like 'sudo' to help test. Fortunately, all the data for both containers was 100% still intact as it was mounted using the volume parameter from the docker-compose file. This means, in theory, you can just nuke the containers, rebuild new containers with the same volume points, and get back in business. 

At this point, I want to stop and say that while this is possible and worked for me, be very very very careful that the data is still intact for both the database and NextCloud instances. If anything beyond the defaults in the docker-compose file was changed - i.e. additional packages or other things were added in the container - the next steps will wipe all of that. 

Steps I used:

Stop the containers:

  sudo docker stop nxtcloud nxtclouddb 

Note: Container names are defined as such in my version of the docker-compose file. Running sudo docker ps -a will show the current names of all the containers.

Backup the container data directories to somewhere safe. Leave the data intact in the original location. 

Note: If you can't get the containers restarted and need to truly nuke and pave, all the files in NextCloud are stored under the NextCloud data directory. On the container this would be /var/www/html/data/, on the local volume it would be under the <NextCloud install directory>/data/data/ on the host system using my docker-compose file.

Delete the containers:
  sudo docker rm nxtcloud nxtclouddb

Download the latest version of the NextCloud image:
 sudo docker image pull nextcloud:latest

Update the docker-compose file to specify the MariaDB version as 10.5:
 image: mariadb:10.5

This step is run because on my first go, NextCloud had issues with the latest version of MariaDB (version 10.6 I believe). The docker-compose pulled the latest image version by default.

Recreate and run the containers. From the directory of the docker-compose and data directories run:
 sudo docker-compose up -d

Once the container names appear, verify they are running correctly:
 sudo docker ps

Now open to the NextCloud browser page. In my case it asked to update the NextCloud instance. This triggered everything to get sorted out in NextCloud, and I could log in and everything was fine. Users can log in, data was intact, and things were copacetic.

The updated version of my NextCloud docker-compose file to reflect the change to use MariaDB version 10.5 is HERE.

Moral of the story, always stop running services on a server, especially the Docker containers BEFORE making a hardware update to the system. No matter how simple it may seem, put the host system in maintenance mode first.

More information on /etc/fstab configs is available here:
  https://linuxhint.com/write-edit-etc-fstab/
  https://help.ubuntu.com/community/Fstab

Hopefully this helps demonstrate how volumes can be used to keep container data persistent, along with the flexibility of working with containers. I was lucky that the upgrade was able to trigger a re-sync of the NextCloud instance and the database. Some more information about doing this manually is in the NextCloud Doc page.
  https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/occ_command.html



Practical Programming using Python with Files

 

This tutorial demonstrates how to use Python to open and read from files in a local computer. Python is a powerful language, and the relative ease it has in opening and reading from text files makes it ideal for doing simple things (like calculating an average as showed), to more advanced (say updating 1000s of files in a single batch, or comparing different output from a research survey). There is an endless number of useful things Python can help automate, and it doesn't take a full time developer or software engineer to use.

The source code that was created is below. 

AverageNumberOfLines

For those interested as well in how to write to files, the script I used to create the 100 files with random ends is also provided. This script works by reading a sample file (Lorem Ipsum text in the example) and then randomly chops between 10 and 100 lines of text for 100 different test files. 

CreatingTextFiles

I hope it can be helpful for those interested in programming and eager to see some simple, but useful, use cases.


Creating Animations in GIMP

 

This article and video is a bit more artistic / creative compared to more of the recent ones. Most recently we've been exploring server-side services and writing scripts. This article looks to tackle creating GIF animations using GIMP (GNU Image Manipulation Program).

The video above plays pretty quick because I just created a very simple scene for demo purposes. The possibilities are really infinite. You could use those same steps to make a banner add for your company, or create a web animation that plays on load of a website. It could also be used for doing some animations with Apps like on iOS or Android (though GIFs in Android need a webview to run). Even something as simple as just quick UI or text tutorials can be done - that's how I made the simple animations in my BASH script blogs. 

Hope this can be helpful for some of you out there. I think GIMP is a wonderful tool - unfortunate name, but wonderful tool - that really can do some outstanding photo and image manipulation. It's also open source and free to use on Windows, Linux and MacOS so if you learn it once you can use it anywhere.

Final image from the tutorial - playing around with the number of frames and delay between frames (currently 180 ms) can create a smoother playback.

SpaceAnimation