Coding Journal

Jordan Vidrine

An experienced business owner + musician with a demonstrated history of management in the Energy sector and an insatiable appetite for learning new and complex skills. Transitioning from an ownership role & seeking employment in Web Development as a Jr Full Stack Developer.

August 5th 2019

Today I’ll be getting back to the NodeJS course I am working through on Udemy. I feel like I have learned a really good amount of great information from this course. I’ve been able to personally use every piece of knowledge from this course into various personal apps and experiments since I started the course.

Sorting, Pagination, & Filtering in Mongoose/MongoDB

The first thing we implemented today was the ability for the database to use timestamps. This was super simple and only involved adding an options object as an argument when creating the Schema for a model.

{
  timestamps: true,
}

What this does is add a createdAt and updatedAt field to each item created in the database.

Filtering Data

To do this, we will setup options for any consumer of our API to use. We will do this by manipulating the GET route on the Task router.

We will implement a query option to pass to the route. That query can be either true or false and will be passed this way: url/tasks?completed=true. We will edit our GET route to use the query to get only certain tasks. We will do this by using passing in a match option on the populate function.

router.get('/tasks', auth, async (req,res) => {
  const match = {};

  if (req.query.completed) {
    match.completed = req.query.completed === 'true'
  }

  try {
    await req.user.populate({
      path: 'tasks',
      match
    }).execPopulate();
    res.send(req.user.tasks)
  } catch (e) {
    res.status(500).send(e)
  }
})

Paginating Data

Pagination is great for implementing the ability to NOT get all data stored in a database at once. This helps load time, especially with databases with lots of information.

To enable this, we will implement two more options into our GET request for the tasks route. These options are limit and skip For instance:

/tasks?limit=10&skip=0 // will return the first 10 items
/tasks?limit=10&skip=10 // will return the second page of 10 items

To implement this into our code, all we need to do is make some changes to the populate method on our user.

await req.user.populate({
  path: 'tasks',
  match,
  options: {
    limit: parseInt(req.query.limit),
    skip: parseInt(req.query.skip)
  }
}).execPopulate();

Sorting Data

The next thing to implement is sorting our data. We can sort in many different ways but for now we will focus on sorting by the created date, and either showing that in an ascending or descending order. The query will look similar to this.

/tasks?sortBy=createdAt:asc
// this would sort the data by when they were created in an ascending way

In order to get this to work properly we will do something to sort that we did to the match object. Depending on what is passed by the user, it will create an object with that sort options, as well as figure out if it needs to ascend or descend.

const sort = {};

if (req.query.sortBy) {
  const parts = req.query.sortBy.split(':')
  sort[parts[0]] = parts[1] === 'desc' ? -1 : 1
}
  await req.user.populate({
    path: 'tasks',
    match,
    options: {
      limit: parseInt(req.query.limit),
      skip: parseInt(req.query.skip),
      sort
    }

This section was fairly simple and straightfoward. I am sure there are lots of ways to sort and paginate through a database, but I am eager to implement this into the drawing prompt app eventually.

Uploading Files with Node + Mongoose

Adding file uploads to express requires an NPM package, that happens to be maintained by the same people maintaining express. This package is called Multer.

Here is the example usage we are given.

// require the library
const multer = require('multer')
// creates a new instance of multer named upload
// when a file is uploaded, multer will store it in a folder called 'images'
const upload = multer({
  dest: 'images'
})
// this specifies to use the method upload.single('upload') as middleware for the apps post request
// when making the request, they key for the file upload needs to match they key in the .single() method
app.post('/upload', upload.single('upload'), (req,res) => {
  res.send()
})

Validating File uploads

We want to validate file size and file type. To do this, we can pass some options to the multer instance we have created.

const upload = multer({
  dest: 'images',
  limits: {
    fileSize: 1000000,

  },
  // here we add a function to filter the uploaded file. We are given 3 arguments we can use.
  // We will be using the file argument, which will be an object, to figure out of the file
  // being uploaded is a Word doc.

  // the cb is the callback to run once we are done filtering.
  fileFilter(req, file, cb) {
    // regex match for doc or docx
    if (!file.originalname.match(/\.(doc|docx)$/)) {
      // this only will run if the filename does NOT end with .doc or .docx
      // we send an error through calling the cb
      return cb(new Error('Please upload a Word document'))
    }
    // this is how you call cb when the fileFilter has passed its test
    cb(undefined,true)


    // Here are three different ways that cb can be called
    //
    // cb(new Error('File must be a PDF'))
    // cb(undefined, true)
    // cb(undefined, false)
  }
})

This is the end of todays learning! I will be setting up and finalizing all of the details of my Art Spark performance this week, so I wont be spending too much timing learning to code. I’ll be back at it next week.