Artificial Intelligence: Type of environments

Let’s first describe what is an agent in artificial intelligence. An intelligent agent is an autonomous entity which observes through sensors and acts upon an environment using actuators and directs its activity towards achieving goals. Intelligent agents may also learn or use knowledge to achieve their goals. They may be very simple or very complex.

When designing artificial intelligence solutions we need to consider aspects such as the the characteristics of the data (classified, unclassified, …), the nature of learning algorithms (supervised, unsupervised, …) and the nature of the environment on which the AI solution operates. We tend to spend big amounts of time in the first two aspects but it turns out, that the characteristics of the environment are one of the absolutely key elements to determine the right models for an AI solution. Understanding the characteristics of the environment is one of the first tasks that we need to do. From this point of view we can consider several categories.

Fully vs Partial observable

An environment is called fully observable if what your agent can sense at any point in time is completely sufficient to make an optimal decision. For example, we can imagina a card game where all the cards are on the table, the momentary site of all those cards is really sufficient to make an optimal choice.

An environment is called partialy observable where you need memory on the side of the agent to make the best possible decision. For example, in the poker game the cards are not openly on the table, and memorizing past moves will help you make a better decision.

Deterministic vs Stochastic

A deterministic environment is one where your agent’s actions uniquely detemine the outcome. For example, in the chess game there is really no randomness when you move a piece, the effect of moving a piece is completely predetermined and, no matter where I am going to move the same piece, the outcome is the same.

A stochastic enviroments there is a certain amount of radomness involved. Games that involve a dice, are stochastic. While you can still deterministically move your pieces, the outcome of an action also involves throwing the dice, and you cannot predict it.

Discrete vs Continuous

A discrete environment is one where you have finitely many action choices, and finitely many things you can sense. For example, the chess has finitely many board positions and finately many things you can do.

A continuous environment is one where the space of possible actions or things you could sense may be infinite. In the game of dards, throwing a dard we have infinite ways to angle it and accelerate it.

Benign vs Adversarial

In benign environments, the environment might be random, it might be stochastic, but it has no objective on its own that would contradict the own objective. Weather is benign, it might be ramdon, it might affect the outcome of your actions but it is not really out there to get you.

In adversarial environemnts, the opponent is really out to get you. In the game of chess the enviroment has the goal of defeat you. Obviously, it is much harder to find good actions in adversarial environments where the opponent actively observes you and counteracts what you are trying to achieve than in benign environments.

I have seen a few more classifications or specifications but, more or less, all of them list the same categories or very similar categories.

Note: Article based on my notes of the course Intro to Artificial Intelligence | Udacity

Artificial Intelligence: Type of environments

CI: Continuous Integration

Continuous integration (CI) is a practice where team members integrate their code early and often to the main branch or code repository. The objective is to reduce the risk, and sometimes pain, generated when we wait till the end of the sprint or project to do it.

One of the biggest benefits of the CI practices is that it allows us to identify and address possible conflicts as soon as possible with the obvious benefit of saving time during our development. In addition, it reduces the amount of time spent in regresion test fixing bugs because it encorages to have a good set of tests. Plus, it gives a better understaning of the features we are developing and the codebase due to the continuous integration of features in our codebase.

What do we need?

Tests, test, tests… Automatic tests

To get the full benefits of CI, we will need to automate our tests to be able to run them for every change that is made to the repository. And when I say repository, I want to say every branch and not just the main branch. Every branch should run the tests and it should not be merged till they are green, all of them. In this way, we will be able to capture issues early and minimise disruptions to our team.

Types of tests

There are many types of tests that can be implemented. We can start small and grow progressibely our coverage. The more meaningfull tests we have the better but, we are running a project, we should find a balance between releasing features and increasing our coverage.

How many should I implement?

To decide about that, we just need to remember two things. The first one is that we want meaninful tests, we should not care about the number of tests we should care about how useful are they. We should write enought tests to be confident that if we introduce a bug (technical or business) we are going to detect it. And second, we should take a look to “The Testing pyramid“, here you can find a link to an artiche of Martin Fowler. Basically, explaing from a cost-efective comparation point of view the amount and type of tests we should write.

Running your tests automatically

One of the things we have discussed we need, it is to run our tests on every change that gets pushed. To do so, we will need to have a service that can monitor our repository and listen to new pushes to the codebase. There are multiple solutions, both, on-premise and in the Cloud.

There are a few considerations we need to think about when we are trying to evaluate a solution like: Platform, resources, access to our repositories, … Some examples are Travis CI, Bamboo or Jenkins.

Immersion in CI

This is not just a technical change, we need to have in mind when we are trying to addopt CI that it is a cultural change too.

We need to start integrating code more often, creating shorter stories or breaking them in short deliverables, we need to keep always the build green, we need to add test in any story, we can use even refactor tasks to add tests and increase our code coverage. We should write test when we fix bugs and so on.

One group of your team that is going to be affected directly by this change is our QA group. They no longer need to test manually trivial capabilities of our application and they can now dedicate more time to providing tools to support developers as well as help them adopt the right testing strategies. Our QA Engineers will be able to focus on facilitating testing with better tooling and datasets as well as help developers grow in their ability to write better code. They will need to test manually some complex stuff but it will not be their main task anymore.

Quick summary

Juts a quick sumarry of the roadmap to addopt CI, we can list the next points:

  1. Start writing code for the critical parts in your system.
  2. Get a CI system to run our tests after every push.
  3. Pay attention to the culture change. Help our team to understand and to achieve.
  4. Keep the build green.
  5. Write tests as part of every story, every bug and every refactor.
  6. Keep doing 4, 5 and 6.

At the beginning, cultural changes are scary and they feel impossible but the rewards sometimes deserve the effort. A new project, if we have one, it is maybe a good option to start changing our minds and taking a CI approach in the development life cycle. If we start with and existing proyect, start slow, step by step but always going forward. And, we should always remember that, this is not just a technological change, it is a cultural change too, explain, share and be patient.

CI: Continuous Integration

CI, CD and CD

When we talk about moder development practices, we often listen some acronyms among we can find CI and CD when we refer the way we build and release software. CI is pretty straightforward and stands for continuous integration. But CD can either mean continuous delivery or continuous deployment. All these practices have things in common but also, they have some significant differences. We are going to explain these similarities and differences.

Continuous integration

In environments where continuous integration is used, developers merge their changes in the main branch as often as the can. These changes are validated by creating a build and running automated tests against the build. Doing this, we avoid the old times painful releases when everything was merged in the last minute.

Continuous integration practice puts a big emphasis on automation testing to keep a healthy build each time the commits are merged in the main branch warning quickly about possible problems.

Continuous delivery

Continuous delivery is the next step towards the release of your changes. This practice make sure you can release to your customers as often and quickly as you want. This means that on top of having automated your testing, you also have automated your release process and you can deploy your application at any point of time by clicking on a button.

With continuous delivery, you can decide to release daily, weekly, fortnightly, or whatever suits your business requirements. However, if you truly want to get the benefits of continuous delivery, you should deploy to production as soon as possible to make sure that you release small batches, that are easy to troubleshoot in case of problems.

Continuous deployment

But, we can go another step farther, and this step is continuous deployment. With this practice, every change that passes all stages of your production pipeline is released to your customers. There is no human intervention (no clicking a button to deploy), and only a fail in test time will prevent a new change to be deployed to production.

Continuous deployment is an excellent way to accelerate the feedback loop with your customers and take pressure off the team as there is not a ‘release day’ anymore. Developers can focus on building software, and they see their work go live minutes after they have finished working on it. Basically, when a developer merges a commit in the main branch, this branch is build, tested and, if everything goes well, deployed to production environments.

Can I use all of them together?

Of course you can, as I have said, each one of them its just a step closer to the production environment. You can set your continuous integration environment, after that, once the team is comfortable, you can add continuous delivery and, finally, continuous deployment can be added to the picture.

Example of CI, CD and CD pipeline

Is it worth it?

Continuous integration:

What it needs from you:

  • Your team will need to write automated tests for each new feature, improvement or bug fix.
  • You need a continuous integration server that can monitor the main repository and run the tests automatically for every new commits pushed.
  • Developers need to merge their changes as often as possible, at least once a day.

What it gives to you:

  • Less bugs get shipped to production as regressions are captured early by the automated tests.
  • Building the release is easy as all integration issues have been solved early.
  • Less context switching as developers are alerted as soon as they break the build and can work on fixing it before they move to another task.
  • Testing costs are reduced drastically – your CI server can run hundreds of tests in the matter of seconds.
  • Your QA team spend less time testing and can focus on significant improvements to the quality culture.

Continuous delivery

What it needs from you:

  • You need a strong foundation in continuous integration and your test suite needs to cover enough of your codebase.
  • Deployments need to be automated. The trigger is still manual but once a deployment is started there should not be a need for human intervention.
  • Your team will most likely need to embrace feature flags so that incomplete features do not affect customers in production.

What it gives to you:

  • The complexity of deploying software has been taken away. Your team does not have to spend days preparing for a release anymore.
  • You can release more often, thus accelerating the feedback loop with your customers.
  • There is much less pressure on decisions for small changes, hence encouraging iterating faster.

Continuous deployment

What it needs from you:

  • Your testing culture needs to be at its best. The quality of your test suite will determine the quality of your releases.
  • Your documentation process will need to keep up with the pace of deployments.
  • Feature flags become an inherent part of the process of releasing significant changes to make sure you can coordinate with other departments (Support, Marketing, PR…).

What it gives to you:

  • You can develop faster as there is no need to pause development for releases. Deployments pipelines are triggered automatically for every change.
  • Releases are less risky and easier to fix in case of problem as you deploy small batches of changes.
  • Customers see a continuous stream of improvements, and quality increases every day, instead of every month, quarter or year.

As said before, you can adopt continuous integration, continuous delivery and continuous deployment. How you do it depends on your needs and your situation. If you are just starting a project and you do not have customers yet you can go for it and implement the three of them and just iterate on them at the same time you iterate on your project and your needs grow. If you have already a project in production you can just go step by step and adopting the practices first in your staging environments.

CI, CD and CD

Advantages of Cloud Computing

Nowadays, it is clear that cloud computing has revolutionized how technology is obtained, used and managed. And all of this has changed how organizations budget and pay for technology services.

Cloud Computing has given us the ability to reconfigure quickly our environments to be able to adapt them to changing business requirements. We can run cost effective services that can scale up and down depending on usage or business demands and, all of this, using pay-per-use billing. Making unnecessary huge upfront infrastructure expenses for enterprises, and balancing the possibilities between been a big enterprise or a new one.

There are multiple and diverse advantages, and most of them depend on the enterprises, the business and the needs they have. But, there are six of them that tend to appear in every case:

Variable vs. Capital Expense

Instead of having to invest in data centers and servers before knowing how they are going to be used, they can be paid when you consume computing resources and paid only for how much you consume.

Economies of Scale

By, using cloud computing, we can achieve a lower variable costs that we would get on our own. Cloud providers like AWS can aggregate hundred of thousands of customers in the cloud, achieving higher economies of scale, which translates in lower prices.

Stop Guessing Capacity

When we make a capacity decision prior to deploying an application, we usually end up either setting expensive idle resources or dealing with limited capacity. With cloud computing there is no more need for guessing. The cloud allow us to access as much or as little as we need to cover our business needs and to scale up or down as required without advanced planning, with in minutes.

Increase Speed and Agility

Deploy new resources to cover new business cases or implementation of prototypes and POCs to experiment now can be achieved with a few clicks provisioning new resources in a simple, easy and fast way usually reducing costs and time and allowing companies to adapt or explore.

Focus on Business Differentiators

Cloud computing allows enterprises to focus on their business priorities, instead of on the heavy lifting of racking, stacking, and powering servers. This allows enterprises to focus on projects that differentiate their businesses.

Go Global in Minutes

With cloud computing enterprises can easily deploy their applications to multiple locations around the world. This allows to provide redundancy and lower latencies. This is not any more reserved just for the largest enterprises. Cloud computing has democratized this ability

Advantages of Cloud Computing

Customizing .vimrc

The vimrc file contains optional runtime configuration settings to initialize Vim when it starts. We can customize Vim by putting suitable commands in your vimrc. We can locate the file in the home directory.

There are some very complex configurations we can do in the vimrc file, but I am going to show just a few simple ones because I usually use IDEs (IntelliJ, Eclipse, Netbens, …) or text editors (Sublime, Notepad++, …) to write my code and I just use Vim when I am connected to a remote server through SSH or locally for a very simple changes in configuration files or similar.

An example of vimdc file:

"Avoid console bell when errors 
set noerrorbells
"Language for messages
set helplang=en
"Line numbers
set number
"Syntax with colors
syntax on
"Parenthesis, brackets and curly brackets matching
set showmatch
"Tab size
set tabstop=2
"Use precedent indentation
set autoindent
"Incremental search
set incsearch
"Ignore case except uppercase string
set ignorecase smartcase
"Mark search results
set hlsearch
"Background color
"set background=dark
Customizing .vimrc

Install Cloud Foundry CLI in macOS

The easiest way to install Cloud Foundry in a macOS system is to use the homebrew package manager.

To install homebrew, we just need to execute the next line in our Terminal:

/usr/bin/ruby -e "$(curl -fsSL"

Once installed, you can “tap” the Cloud Foundry repository:

brew tap cloudfoundry/tap

Finally, you install the Cloud Foundry CLI with:

brew install cloudfoundry/tap/cf-cli

Once you have installed the CLI tool, you should be able to verify that it works, by opening a Terminal and running:

cf --version

This should show something like:

cf version 6.30.0+decf883fc.2017-09-01

If you see a result like this, the CLI is installed correctly and you can start playing.

Now, we need a trial account in a Cloud Foundry provider. There are multiple option we can check in the Cloud Foundry Certified Providers page. Once we have created the account we can proceed to login with out CLI.

cf login
API endpoint>


Targeted org example

Targeted space dev

API endpoint: (API version: 2.75.0)
Org:          example
Space:        dev

In the above output, the email is the address you used to sign up for a service.

Once you have successfully logged in, you are authenticated with the system and the Cloud Foundry provider you use knows what information you can access and what your account can do.

The CLI tool stores some of this information, the Cloud Foundry Endpoint API, and a “token” given when you authenticated. When you logged in, instead of saving your password, Cloud Foundry generated a temporary token that the CLI can store. The CLI then can use this token instead of asking for your email and password again for every command.

The token will expire, usually in 24 hours, and the CLI will need you to login again. When you do, it will remember the last API Endpoint you used, so you now only have to provide your email and password to re-authenticate for another 24 hours.

First commands

  • cf help: Shows CLI help.
  • cf help <command>: Shows CLI help for an specific command.
  • cf <command> –help: Shows CLI help for an specific command.
  • cf help -a: Lists all the commands available in the CLI.
Install Cloud Foundry CLI in macOS

Microservices: Capability model

Microservices is an area that is still evolving. There is no standard or reference architecture for microservices. Some of the architectures available publicly nowadays are from vendors and, obviously, they try to promote their own tools stack.

But, even, do not having an specific standard or reference we can sketch out some guidelines to design and develop microservices.

Capability Model
Image 1. Capability Model (Seen in “Spring 5.0 Microservices – Second Edition)

As we can see, the capability model is main splitted in four different areas:

  • Core capabilities (per microservice).
  • Supporting capabilities.
  • Process and governance capabilities.
  • Infrastructure capabilities.

Core capabilities

Core capabilities are those components generally packaged inside a single microservice. In case of microservices and fat jars approach, everything will be inside the file we are generating.

Service listeners and libraries

This box is referring to the listener and libraries the microservice has in place to accept service requests. The can be HTTP listeners, message listeners or more. There is one exception though, if the microservice is in char only of scheduled tasks, maybe, it does not need listeners.

Storage capability

Microservices can have some king of storage to do properly their task, physical storages like MySQ, MongoDB or Elasticsearch, or in-memory storages, caches or in-memory data grids like Ehcache, Hazelcast or others. There are some different storages but, it does not matter what type of storage is used, this will be owned by the microservice.

Service implementation

This is were the business logic is implemented. It should follow tradicional design approaches like modularization and multi-layered. Different microservices can be implemented in different languages and, as a recommendation, they should be as stateless as possible.

Service endpoint

This box just refers to the external APIs offered by the microservice. Both included, asynchronous and synchronous, been possible to use technologies from REST/JSON to messaging.

Infrastructure capabilities

To deploy our application and for the application to work properly we need some infrastructure and infrastructure management capabilities.


For obvious reason, microservice architectures fit more in cloud based environments that in tradicional data center environments. Things like scaling, cost effective management and the cost of the physical infrastructures and the operations make in multiple occasion a cloud solution more cost effective.

We can find different providers like AWS, Azure or IBM Bluemix.

Container runtime

There are multiple options here and, obviously, container solutions are not the only solutions. There are option like virtual machines but, from a resources point of view, the last ones consume more of them. In addition, usually it is much faster to start an instance of a new container than to start a new virtual machine.

Here, we can find technologies like Docker, Rocket and LXD.

Container orchestration

One of the challenges in the microservices world is that the number of instances, containers or virtual machines grows adding complexity, if not making it impossible, manual provisioning and deployments. Here is were containers orchestration tools like Kubernetes, Mesos or Marathon come quite handy, helping us to automatically deploy applications, adjust traffic flows and replicate instance among other.

Supporting capabilities

They are not related with the microservices world but they are essential for supporting large systems.

Service gateway

The service gateway help us with the routing, policy enforcement, a proxy for our services or composing multiple service endpoints. There are some options one of them is the Spring Cloud Zuul gateway.

Software defined load balancer

Our load balancers should be smart enough to be able to manage situations where new instances are added or removed, in general, when there are changes in the topology.

There are a few solutions, one of them is the combination of Ribbon, Eureka and Zuul in Spring Cloud Netflix. A second one can be Marathon Load Balancer.

Central log management

When the number of microservices grow in our system the different operations that before were in one server now are taking place in multiple server. All this servers produce logs and to have them in different machines make quite difficult to debug errors sometimes. For this reason, we should have a centrally-managed log repository. In addition, all the generated logs should have a correlation ID to be able to track an execution easily.

Service discovery

With the amount of services increasing the static service resolution is close to imposible. To support all the new addition, we need a service discovery that can deal with this situation in runtime. One option is Spring Cloud Eureka. A different one, more focus in container discovery is Mesos.

Security service

Monolithic applications were able to manage security themselves but, in a microservices ecosystem we need authentication and token services to allow all the communications flow in our ecosystem.

Spring offers a couple of solution like Spring OAuth or Spring Security, but any single sign-on solution should be good.

Service configuration

As we said int he previous article, configurations should be externalized. It is an interesting choice set in our environments and configuration server. Spring, again, provides Spring Cloud Config but there are some other alternatives.

Ops monitoring

There is need to remember that now, with all this new instances, all of them scaling up and down, environment changes, service dependencies and new deployments going on, one of the most important things it is to monitor our system.  Things like Spring Cloud Netflix Turbine or Hystrix dashboard provide service-level information. There are some other tools that provide end-to-end monitoring like AppDynamic or NewRelic.

Dependency management

It is recommended the use of some dependency management visualization tools to be aware of the system complexity. They will help us to check dependencies among services and to take appropriate design decisions.

Data lake

As we have said before, each microservice should have each own data storage and this should not be shared between different microservices. From a design point of view, this is a great solution but, sometimes, organizations need to create reports or they have some business process that use data from different services. To avoid unnecessary dependencies we can set a data lake. They are like data warehouses where to store raw data without any assumption about how the information is going to be use. In this way, any service that needs information about another service, just needs to go to the data lake to find the data.

On of the things we need to consider in this approach is that we ned to propagate the changes to the data lake to maintain the information in synch, some tools that can help us with this is Spring Cloud Data Flow or Kafka.

Reliable messaging

We want to maximize the decoupling among microservices. The way to do this is to develop them as much reactive as possible. For this reliable messaging system are needed. Tools like RabbitMQ, ActiveMQ or Kafka are good for this purpose.

Process and governance capabilities

Basically, how we put everything together and we survive. We need some processes, tool and guidelines around microservices implementations.


One of the keys about using a microservice oriented architecture is been agile, quick deploys, builds, continuous integrations, testing… Here is where a DevOps culture come handy in opposite to the waterfall culture.

Automation tools

Continuous integration, continuous delivery, continuous deployments, test automation, all of them are needed or at least recommended in a microservices environment.

And again, testing, testing, testing. I cannot say how important in this, now that we have our system splitted in microservices the need to use mocking techniques to test, and to be completely confident, we need functional and integration tests.

Container registry

We are going to create containers, in the same way we need a repository to store the artifacts we build, we need a container registry to store our containers. There are some options like Docker Hub, Google Container Repository or Amazon EC2 container registry.

Microservice documentation

Microservices system are based on communication. Communication among microservices, calls to APIs offered by this microservices but, we need to ensure that people that want to use our available APIs can understand how to do it. For this reason is important to have a good API repository:

  • Expose repository via a web browser.
  • Provide easy ways to navigate APIs.
  • Well organized.
  • Possibility to invoke and test the endpoint with examples.

For all of this we can use tools like Swagger or RAML.

Reference architecture and libraries

In an ecosystem like this the need to set standard, reference models, best practices and guidelines on how to implement better microservices are even more important than before. All of this should live as a architecture blueprints, libraries, tools and techniques promoted and enforced by the organizations and the developer teams.

I hope that after this article, we start having a rough idea about how to tackle the implementation of our systems following a microservice approach. In addition, a few tools to start playing with.

Note: Article based on my notes about the book “Spring 5.0 Microservices – Second Edition”. Rajesh R. V

Microservices: Capability model