Creating Releases

Releases

Last time on goblin camp adventure we finally got a functional build running, there would be no better way to celebrate than to make a release on github. There are a couple hurdles beforehand that need to be taken care of. First off the code currently in the master branch does not support the libtcod we have travis building it with. Second off we have no way of automatically uploading releases and finally I'm not even sure if it would work on other machines if it was compiled in travis.

Actually Releasing

To have a binary to release first step is to have a working, running, non segfaulting version of the game. This is already done locally on a test vm but it needs to be in github so we can have travis make it automatically for us. I can't just overwrite the scripts in the master to use the old version so I decided to make a branch that would be in charge of making the releases for the time being. That is until master becomes more stable. Appropriatley I called the branch working-build. I then had to change the build scripts to use the older libraries that we working build needed, after that I had to use travis cli tool to generate the secure key that allows it to push to releases. Except when I used cmake on my machine it would make a pkg directory with all the files inside of it nicely bundled, but when I ran it on travis it stalled it to the user directories. After many hail mary commands I found out that I need to run cmake twice to make that folder so I could put it in a tar. I thought I was done, except when I tagged a branch and pushed the tag to github the release wans't there. So I cleaned up some stuff in the script and pushed it again with a tag and the same thing happened, no release. Except now the first tag I made had one. I honestly don't know why this happened but it was good enough for a first ever github goblin camp binary release. Someone will probably fix it later...hopefully

Removing Boost

Why its problematic

Having boost in the project is a big help since it implements a huge range of useful features and objects. However over the years a lot of these features have been moved into the c++ standard library. Having boost included also makes building goblin camp a little harder across platforms. This lead us to start replacing the boost libraries with the std equivalent.

The Move

Moving to std libraries most of the time isn't a problem since some features of boost are transferred into the standard. So for libraries like cstdint it almost exactly the same as boost/cstdint. To change it all that needs to be done is change the namespace and include statement to use the standard one. One pr after and we use one less boost library. I feel like most of these boost libraries won't be too hard to remove except for the one deeply ingrained in the project.

Goblin Camp Continued

Travis continued

This pr addressed that after a lot of trial and error our travis.yml got quite out of hand. My job was to clean it up and make it look like it was made by someone who knows what they're doing. It was kinda hard to implement since every change I was unsure of I had to commit it and push it just so I can see if it worked. Needless to say this made it take a lot longer than I thought. I also moved the depenencies that were being installed from source and put it into a shell script. This removed the clutter in the .travis.yml and the script could be used by people who also want to build them.

The future

What I spent most of my time on was trying to get an actual playable version of goblin camp running. The boost library it needed was out of date and couldn't be installed through my package manager and when I tried to install it from source it couldn't be found by my makefile. Trying to preserve my sanity I just started a ubuntu 14.04 vm and used their package manager to get all the old packages I needed, and surprisingly it worked.

In a way it is slightly terrifying since now I have no excuse but to work on the code...

First PR

First Internal PR

Goblin Camp has been classified internal so I've been working on that trying to get travis ci working with it. Travis is an automated build system that when you create a branch or a pull request will build the project to check for errors. This makes development easier since people who can't build the project or forget to will still have it built. This is useful to me as a maintainer since its a quick way to see if it is safe to merge into master or just test things without having to go through the trouble of setting up the environment.

Travis CI

All of the build configuration is done in the .travis.yml file in the root of the project. I gave it commands to pull packages that were required from the Goblin Camp as well as try to run the build script. Of course it didn't build since the Boost.Build was horribly outdated and trying to find out how to make it build would required a blood sacrifice or something along those lines. Thankfully it was not in the scope of my issue to get the project to build. I just had to make a .travis.yml that tried to build it. It was pretty simple but I learned a lot about travis, like its free and kinda slow and our build script kinda sucks. It finished off this pull request even through right now it's kinda of useless but I have a feeling that trying to get Goblin Camp to build will be a whole different story.

Arcane Knowledge

Goblin Camp

Goblin camp has recently taken a large set of real estate in my head. Its a perfect storm of challenges and things I understand. The first big challenge was getting this beast to build and spoiler warning, we did. It took us a lot longer than I'd like to admit but what I would like to admit is that I didn't do this alone. A good friend of mine, Tony Sim trudged through the perils with me on our journey to compilation. Which unlike most open-source we worked on it in the same room, using the same computer(mine for the record). This encapsules what I think opensource has the capability for, collaborating locally or globally. This very local open source has its own advantages and disadvantages than over the net. First off you can bounce ideas and try things very rapidly as well as tag out when you're head is going under water. This was especially useful since the only dev environment we had that had the potential to run this was my laptop and all the changes we made were instantly visible to both of us.

Hacking through

To try and get it compiled we mostly just threw commands around and installed packages with no regard. Exporting environment variables, reading posts, trying not to give up were all things that we did. Every time we went through this we would gain maybe 1-10% extra compilation then it would crash again. Telling us errors about not being able to locate packages that we're removed from the libraries it uses, saying it can't find packages we have, that our packages are out of date, our compiler is out of date, it can't find headers from our packages, it just went on and ooon. So we hacked away at it for a good few hours or days, I couldn't tell, when you've fallen into the rabbit hole time gets distorted.

Success?

After the whole ordeal we had it at 100% compilation and we thought this was it!!!... then it failed at linking. I have never been more discouraged by a single program. The problem was in a cmake script. It was an easy fix, but hard to find. After it compiles AND links I thought the build script didn't work, in the instructions it says it would put it in the build folder, but I found it in the src or root I can't remember. The moment of truth was finally upon us. We have slain all the enemies the compiler threw at us. We found the sacred texts that allowed us to build every dependency. We shacked the objects together. We were victorious! Our prize? "Segmentation Fault". No GUI no nothing, just "Segmentation Fault".

Pull Request

For my pull request I updated the travis.yml with all the environment magic we struggled to find to get some automated building for it, even if it seg faults. On our journey we found a lot of issues that were made into bugs and then into pr's. Even though we both worked on it, it was easier for one of use to commit the changes to a branch and merge it. My issue was updating travis which is something I was familiar with and I also had a pull request that compiled a package from source. On the horizon we're probably going to faze out the outdated libaries and try to get it to not seg fault, who knows maybe this project is cursed but now I bear the mark of the curse and I've dived too deep to come up for air now. Not when I've gotten this far.

Diving Deeper Into Open Source

Recovery

After the frantic nature of finding projects, finding bugs, building projects, and making pull requests I needed a little break. So I have been looking into projects that might find interesting. A few that caught my interest were Dark Chatter, Creative Collab and supernova from fellow students.

Dark Chatter

Dark chatter is interesting to me since seems like it will rely on low level stuff like constructing packets and it will help me understand more on how wifi works.

Creative Collab

Creative collab also expressed interest in doing the project in mostly as a backend server that has public end points. It would be neat to learn how to create a server to host an api that anyone can use.

supernova

Super nova caught my interest not because of the idea but more of the language choice. I have never used rust and learning it through open source seems like a good way to find out about the language.

External Projects

Goblin Camp

Goblin camp is kind of like a open source dwarf fortress and it was forked by a class mate with the intention of parallelizing it, since the original is still running on a single thread. As of right now I am stuck in the same slog that kept me from trying other projects, getting it to build. But since I have a little more leeway with this due date being more lenient than the hacktoberfest one I think I will try to push through to get it to work. The project is also been abandoned for a couple years so it might take some time to get it all working.

Path Forward

Now I'm going to try to dedicate some time to building the projects I'm interested as well as seeing what I can contribute. Maybe even make some issues, who knows, I feel more confident after hacktoberfest and excited to apply all that I learned going forward.

Hacktoberfest Review

Overall Thoughts

Overall I think hacktoberfest was a neat way to dive into the opensource community. I also found it very frustrating finding a project that I liked or could build. Since there were many times where I found a neat project but couldn't build it and didn't want to waste hours figuring out why. I also found the amount of repositories just made for hacktoberfest pull requests annoying since they would saturate the hacktoberfest tag with what I thought was not a real project, more like a collection of code. Even though all my complaining it is still my fault I didn't contribute to some projects I found interesting. I didn't leave enough time to them and greatly underestimated how long just getting something running would take. Moving forward I'll take this into consideration when I want to work on a project and give my self at least twice the amount of time I would think it would take.

Hacktoberfest PR #5

The Project

The project was the same as the last pull request, I did the same kind of fix as the last one but this time I was adding a dairy tag to all foods that had dairy in them. Most of these foods already had milk in them so I felt it was a bit redundant to add the dairy tag but since the issue was still open I thought I was still a problem. This pull request was a still larger than the last one since there were a lot of lines that needed to be changed. That being said these two last pull request were pretty simple and didn't require to understand a lot of the nitty gritty in the project, which I'm thankful for.

Hacktoberfest PR #4

The Project

This project is Cataclysm - Dark Days Ahead is a turn-based survival game set in a post-apocalyptic world. I was led on to this project by a fellow student who told me there was some easy fixes that I could do. My fix was just editing a recipe in a JSON according to contributors suggestions. This project was a pretty big one at 1.3 gigabytes so I didn't bother building it I just made the pull request and let their automatic build systems check it.

Hacktoberfest PR #3

Third Pull Request

I was browsing thourgh projects and while I was reading the Readme I thought I saw typo so I made a reckless pull request without creating an issue or checking with the maintainers. This led to the pull request being closed because the issue I thought I found was actually not an issue. I learned that even the process of looking through issues/making issues and communication is a vital part of open source and I can't just drop what I think are fixes willy nilly.

Hacktoberfest PR #2

The Project

For my second pull request I just wanted to do a quick and easy fix. I had enough of cloning projects, builds failing, dependencies missing and just decided how about we update a README that sounds like it can be done without install 100000000 files. So I searched around and found the repository for HandBrake, HandBrake is an opensource video transcoder. Their main page had a docs repository which seemed just like the place to look.

Issue

It was actually quite easy finding an issue. It seems like this repo didn't have too much activity and the newest issue a month old. Conveniently the newest issue also didn't have anyone actively working on it so I decided to take a stab at it. It was just two misspelling fixes, created a pull request and that was it.

Conclusion

Even thought this pull request is pretty small I still feel a sense of accomplishment. Handbrake is a widely used tool and its nice to contribute to a larger project.

Hacktoberfest PR #1

First Impressions

Honestly I was excited about hacktoberfest when I first heard about it, I thought it would be full of huge opensource projects and bugs on stuff I use every day, except I found most of the hacktoberfest labeled issues are more of collections of code like a list of nice words or a set of algorithms in different languages. This made searching for actual projects tedious since there was a lot of these kinds of projects making issues but eventually I found a project that struck my interest.

Project

This project was a game engine written in c++ then a python wrapper is put on it so it can be used in python. I found this project captivating since it used combined two languages that I'm interested in. The game engine seemed to be in a primitive but working state. But more I read into the code the more it became confusing, it used a lot of strange calculations and an enormous amount of OpenGL functions which I couldn't understand. I kind of gave up on it since it seemed like a never ending rabbit hole of code. Except I couldn't stop thinking about it, I guess even if I didn't understand the code I still found it interesting.

Issue

At the time I was choosing an issue there was only two open and one of them was already being work on so I was left to do this one. Although I had no understanding of OpenGL or how texture rendering even worked I chipped away at it for a few days, looking at each function call, traversing files just to find some clue on how I could implement the fix. I found the line that used an image from file and the line that loaded it. All I had to do was change that and be done. So I messed around with it and got it working! Even I was surprised when it started showing different a image. This victory was bittersweet because after I changed the loaded image I couldn't find a way to specify that image in the python constructor. Thankfully it was pointed out that the python constructor wasn't in the scope of my pull request so I was in the clear. I put up my pr and that was it.

Pull Request

The project owner brought up some valid criticism in my pull request. I was loading the texture from file every frame which would cause additional lag and I could just have the image stored in a object group since it is being used by every object in that group. I made these changes and then tragedy struck... The pretty colors from the pictures weren't being shown anymore. It seemed like if the image wasn't being loaded right before it was being used, it would get corrupted somewhere along the line. I set sail back into the sea of code and tried to find out why this was but I just couldn't. I replied to the project owners comment telling him my problem seeing if he could find out why this was happening but as of righting this he has not yet replied.

Conclusion

I learned a bit about OpenGL and some game engine logic. I am still interested in this project but as of now it is unclear whether or not I will be worth the time investment and stress trying to contribute to this project. Overall even though it sounds like I had a terrible experience I actually enjoyed sifting through code and finally finding a nugget of gold only to find out it was fools gold.

First Open Source Experience

The First Step

Diving into a open source project was intimidating and frustrating, I spent a few hours just trying to get comfortable with the project space. Then I figured I would delve deep into the source code and try to understand how it all came together. This was... a mistake, not because the code is horribly written or anything it's just that I didn't have the time to learn this whole API. It was like trying to put a big puzzle together, It could be done but I had nowhere near the time or patience to do it. So I settled on porting a stat test to use the new Promise API in fs. This is what I think I should have done in the first place, don't dive straight in maybe just wet my feet and see how it feels. I opened up an issue and away I went.

The Nitty Gritty

The test I chose to port was this one. I had some knowledge of stat from the previous post so I could understand almost all of what was going on in here. First of all I made a branch that was call issue-423 (423 being the number on my issue). It was pretty easy to convert to using promises, just import the library, switch the callback to .then/.catch and that was it, I pushed it to my branch and made my pull request and thought I was finished. Except I had a few bugs that I wasn't aware of. Thankfully open-source community was there to help and they pointed out all the things I could improve. For instance changing my test be more promise-friendly and some style fixes, things that I wouldn't have thought of like returning the whole promise since the it() function can handle them. GitHub also made this process really fluid since all changes that I made in my local branch were reflected in the pull request. Even though I made the initial code I felt everyone who posted reviews helped write it.

The Give Back

After getting good feedback and improving my pull request I thought it was about time I re-payed the favor. I basically took what others suggested about my code and applied it to other pull requests. Like here and here. Overall I feel like not only my code got better but I helped other people learn as well. Working in open-source made me fix things that normally I would be too lazy to, it also helped motivate me learn git which I struggled with for quite some time.

NodeJS Stat

The Basics

If you're not familiar with stat I will give you a quick rundown. Stat is used in UNIX to view information about a file. In the fs module of NodeJs most stat related functions return a Stats object. This is what is looks like in the documentation.

Stats {
  dev: 2114,
  ino: 48064969,
  mode: 33188,
  nlink: 1,
  uid: 85,
  gid: 100,
  rdev: 0,
  size: 527,
  blksize: 4096,
  blocks: 8,
  atimeMs: 1318289051000.1,
  mtimeMs: 1318289051000.1,
  ctimeMs: 1318289051000.1,
  birthtimeMs: 1318289051000.1,
  atime: Mon, 10 Oct 2011 23:24:11 GMT,
  mtime: Mon, 10 Oct 2011 23:24:11 GMT,
  ctime: Mon, 10 Oct 2011 23:24:11 GMT,
  birthtime: Mon, 10 Oct 2011 23:24:11 GMT } 

This object holds a lot of information that seems to be irrelevant to us, but in reality it is all useful! I made a handy reference sheet so that the file statistics represent

 
  dev: Device ID
  ino: Inode number
  mode: File type and mode
  nlink: Number of links
  uid: User ID of owner
  gid: Group ID of owner
  rdev: Device ID (if special file)
  size: Total size in bytes
  blksize: Block size of file system I/O
  blocks: Number of 512 byte blocks allocated
  atimeMs: Time of last access in milliseconds
  mtimeMs: Time of last modification in milliseconds
  ctimeMs: Time of last status change in milliseconds
  birthtimeMs: Time of creation in milliseconds
  atime: Time of last access 
  mtime: Time of last modification
  ctime: Time of last status
  birthtime: Time of creation 

Lets put our new knowledge to use.

const fs = require("fs");
let filepath = "test1.txt";
fs.stat(filepath, (err, stats) => {
    if (err) {
        console.log(err);
    } else {
        console.log("Date Created: ");
        console.log(stats.birthtime);
        console.log("Date Accessed: ");
        console.log(stats.atime);
        console.log("Date Modified: ");
        console.log(stats.mtime);
        console.log("Date Changed: ");
        console.log(stats.ctime);
    }
});

Output:

Date Created: 
2018-09-13T22:04:58.180Z
Date Accessed: 
2018-09-13T22:04:58.180Z
Date Modified: 
2018-09-13T22:04:58.180Z
Date Changed: 
2018-09-13T22:04:58.180Z

We can also do the same thing using fsPromises

const fsPromises = require("fs").promises;
let filepath = "test1.txt";
let statPromise = fsPromises.stat(filepath);
statPromise.then((stats) => {
        console.log("Date Created: ");
        console.log(stats.birthtime);
        console.log("Date Accessed: ");
        console.log(stats.atime);
        console.log("Date Modified: ");
        console.log(stats.mtime);
        console.log("Date Changed: ");
        console.log(stats.ctime);
    }).catch((err) => {
        console.log(err); 
});

lstat works similarly to stat except if the filepath is a symbolic link the link in statted instead of the file. fstat takes a file descriptor instead of a filepath. To use fstat open a file using fs then using the file descrpitor from the opfunction and pass it into fstat
It would look something like this:

const fs = require("fs");
let filepath = "test1.txt";
fs.open(filepath, 'r', (err, fd) =>{
      if (err) console.log(err);
      console.log(fd);
      fs.fstat(fd, (err, stats) => {
          if (err) {
              console.log(err);
          } else {
              console.log(stats);
          }
      });
      // always close the file descriptor!
      fs.close(fd, (err) => {
          if (err) throw err;
      });
 });