Types of dependencies in NPM.

Introduction

When we start a new javascript project and we count on adding a minimal level of complexity to it, we will eventually need to make use of external libraries if we want to avoid writing every feature totally from scratch. For that matter, you can choose to take three approaches:

  • Download the library code bundle and adapt it to your development and deployment process. This is easy if the library is very small, you don't care about updating it's version, and you know very specifically what you want from it.
  • Use a Content Delivery Network to access a remote copy of the library hosted in the cloud.
  • Use a tool like NPM, Bower, or the new kid in town Yarn. This are package managers: pieces of software that allow the developer manage the projet's dependencies by fetching them from a central repository, specify each library's desired version and execute scripts for different steps of the development process.

When you are working on a project and decide to use npm as it's package manager, it will use a file named package.json in the root of the project's directory. This file contains the information about the dependencies on external libraries it has, along with other useful information. It sits as the point of reference that npm will use when fetching and updating your dependencies. More information about the package.json file here.

How does it work.

Very briefly explained, npm will read the contents of your package.json file and will enumerate the libraries your current project depends on. It will then fetch each library in a recursive manner, namely, each one of those libraries may contain themselves a package.json file enumerating the dependencies that library relies upon, so it'll fetch them again, and so on, and so on.

Dependencies are expressed in a hash with a pair of

{
  "package_name_1": "version range",
  "package_name_2": "version range",
  ...
}

per each of them. The version range can be a string indicating the version range to fetch from the central repository, a tarball or a git url pointing to a specific tag or commit.

There are different ways inside the package.json file to indicate the nature of the dependencies: dependencies, devDependencies and peerDependencies.
You can also find bundledDependencies and optionalDependencies, but I do not see them used that often.

dependencies

These are the libraries your project directly depends on in order to properly execute. There is a direct dependency from the correct functioning of your application to the fetching of this packages. After compiling and minifying your javascript, the final bundle must include these libraries code to work.

devDependencies

They represent the packages needed during the development of your project, such as tools and frameworks, that are not needed in the final product. An example for a project developed with the Ember javascript framework, could be:

"devDependencies": {
  "ember-cli": "2.11.1",
  "ember-cli-sass": "^5.3.1",
}
  • ember-cli is the command line tool that helps the creation of blueprints for the project, run scripts, etc.. much as the rails cli tool.
  • ember-cli-sass contains files to help translate the sass code into css code.

As you can see, a final user of your product is not interested to have this auxiliary tools in the final bundle.

peerDependencies

peerDependencies are used to avoid name clashes and application crashes when a package depends on a library's version that is not the same as the specified one in the project's package.json. For example:

{
  "name": "test_peer",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "grunt-webpack": "^3.0.0",
    "webpack": "^1.0.0"
  }
}

I specify version "1" of webpack and also install grunt-webpack. grunt-webpack states a different webpack version in it's peerDependencies hash:

"peerDependencies": {
  "webpack": "^2.1.0-beta || ^2.2.0-rc || 2 || 3"
},

The result of npm install this project is:

test_peer@1.0.0 /home/esteban/gits/test/test_peer  
├─┬ grunt-webpack@3.0.0
│ ├─┬ deep-for-each@1.0.6
│ │ └─┬ is-plain-object@2.0.3
│ │   └── isobject@3.0.0
│ └── lodash@4.17.4
└─┬ UNMET PEER DEPENDENCY webpack@1.0.0

... etc ...

This means that while in my project I downloaded version 1 of webpack, grunt-webpack depends on another set of versions to safely execute, so the package manager will intercede for us and warn us that this setup may break the program.

bundledDependencies

They can also be written as bundleDependencies. They enumerate the dependencies that must be included in your project just as regular dependencies. They will also be included in the tarball generated as the result of executing npm pack.

They are used for example if you want to include third party libraries that are not part of the npm registry, or distribute your project as a module.

optionalDependencies

Just like regular dependencies, but builds don't fail if the dependencies are not met.

That's it, I hope it was a little bit more clarifying for readers!

References:

Easy way to log network traffic from docker containers

Introduction

Let's say that you are working on a project using docker and running several containers in the host network, or you have multiple containers leveraging the power of the docker-compose script and running concurrently sharing a single docker network. Now for instance, imagine that you experience some slowness while they are executing, and you suspect one or several of them are misusing the network causing a performance bottleneck, and you want to identify it. Another case could be a scenario where you want to extract network traffic's statistics and plot them afterwards using tcptrace and gnuplot.

It would be handy to have a tool that automated the process of detecting new containers added to a given docker network, start watching their traffic and dump it into .pcap files following a structured format.

I wrote the docker-network script to provide this piece of functionality. You can get the script here.

How

You just need to clone the script and execute it once you have started your containers.

$ ./docker-watcher.sh -d|--discovery <discovery_time> -t|--t <time_period> -r|--rotate <yes/no> -n|--network <network> (default: all) -o|--output <output_folder>
  • discovery_period: is the interval time for the script to wait to check if new containers have been started.
  • time_period: is the elapsed time to store new traffic dumps.
  • rotate: whether or not to rotate logs
  • network: the docker network to observe
  • output_folder: the output folder where pcaps will be stored.

docker-watcher's functioning is very easy. It will watch for new containers added into the specified docker network and start a new tcpdump process instructing it to listen into the target docker network's interface and filtering the traffic by container.

For each container one or several .pcap dump files will be created, depending on whether or not you want to rotate logs.

Here you can see an example of running a mysql docker container and starting to watch traffic for it:

Considerations

In order to extract information about the docker containers, the script uses Go templates and heavily depends on the format of the information output when performing docker inspect to a container. Changes in the structure of the information would break the tool, needing to adapt it to the new structure.

The latest docker version it supports is:

Client:  
 Version:      17.05.0-ce
 API version:  1.29
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:10:54 2017
 OS/Arch:      linux/amd64

Server:  
 Version:      17.05.0-ce
 API version:  1.29 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:10:54 2017
 OS/Arch:      linux/amd64
 Experimental: false

Minimal working setup Rails 5 API + ActiveAdmin

First of all add the ActiveAdmin and the Devise gems. Devise is needed because ActiveAdmin will use it as the authentication mechanism.

Gemfile:

gem 'devise', '> 4.x'  
gem 'activeadmin', github: 'activeadmin'  

We install the gems:

bundle install  

Then we modify several files in order to adapt the Rails API application:

app/controllers/application_controller.rb:

  • ActionController::API changed to ActionController::Base
  • Added protection against XSRF attacks.
class ApplicationController < ActionController::Base  
  include ActionController::Serialization

  protect_from_forgery :with => :exception
end  

config/application.rb: We need to change config.api_only to false in order to enable several middleware needed to make Devise work. Otherwise it will always redirect to login page by default.

module YourApp  
  class Application < Rails::Application
    config.api_only = false
  end
end  

Install the ActiveAdmin system in the application (this will create an AdminUser model):

rails generate active_admin:install  

If you want to use an existing model as admin run:

rails generate active_admin:install User  

Create the tables and seed the data:

rails db:migrate  
rails db:seed  

This is the bare minimum that I needed to successfully have the admin page and log into the dashboard.

Ember.run.later and setTimeout comparing scenario.

In the Ember internals (Backburner) it can be found that Ember.run.later basically calls setTimeout() under the hood, but respecting the firing order by which other timeouts were added into the queue.

import Ember from 'ember';

export default Ember.Controller.extend({  
  appName: "Timers comparison",
  init: function() {
    this._super();
    this.set('myErrCountLater', 0);
    this.set('myErrCountTimeout', 0);
    this.myFunc(0);
    this.myOtherFunc(0);
  },
  myFunc: function(errorCount) {
    if (errorCount < 7) {
      self = this;
            this.set('myErrCountLater', errorCount);
        console.log("Error Count Later is " + errorCount + ", repeating");
      Ember.run.later(this, function () {
        return self.myFunc(errorCount+1);
      }, 1500);
    }
    else {
        console.log("Everything failed");
    }
    },
  myOtherFunc: function(errorCount) {
    if (errorCount < 7) {
      self = this;
      this.set('myErrCountTimeout', errorCount)
      console.log("Error Count Timeout is " + errorCount + ", repeating");
      setTimeout(function() {
        return self.myOtherFunc(errorCount + 1);
      }, 1500);
    }
    else {
      console.log("Everything failed");
    }
  }
});

Internally the only difference is that every function that is run as a callback inside Ember.run.later will be pushed into an array where it will fire only after all previously queued events have fired, respecting the queuing order.

Hello World!

Why?

Once again I moved from my old half-handmade custom website, and my blog powered by the Nibbleblog engine, to this fancy Ghost instance with a custom theme, recommended by Eugenio Estrada.

All this is with the intent of structuring a little better my online presence and not having to fiddle around with an outdated and discontinued blog engine, in addition of the facewash I wanted to apply into my personal site(brand).

I know this current design may not be the best eye candy ever, but I like the sobriety, clarity and simplicity of it. Since I read this talk's transcription I have been paying attention to web performance, user experience and design with special care.

I know I could have installed by hand a simple html skeleton and sprinkle some javascript on top of it. And I specially notice the irony on having mentioned web obesity previously and then rolling my site here, but I am willing to accept the tradeoff of such a nice and simple to use blogging platform as Ghost and therefore sending a bigger payload to users than to lose any more time in configuration and deployment of a slimmer blog engine.

Furthermore, this blog as always will serve as my personal hodgepodge of rants, reflections and technical blog posts that will only represent my and only my opinions.

Without further ado,

Follow the rabbit hole!