Entradas

Mostrando entradas de 2017

Decorators/wrappers

Decorators/wrappers  What is it? 1 Related concepts 2 First class function (FCC) 3 References 4 Closure 4 When do we have a closure? 5 References 5 Videos 5 Blogs 5  What is it? In object-oriented programming , the decorator pattern (also known as Wrapper, an alternative naming shared with the Adapter pattern ) is a design pattern that allows behavior to be added to an individual object , either statically or dynamically, without affecting the behavior of other objects from the same class . For example and motivation see: https://en.wikipedia.org/wiki/Decorator_pattern#Motivation Example for my case: # Decorators from functools import wraps def my_logger(orig_func):    import logging    logging.basicConfig(filename='{}.log'.format(orig_func.__name__), level=logging.INFO)    @wraps(orig_func)    def wrapper(*args, **kwargs):        logging.info(            'Ran with args: {}, and kwargs: {}'.format(args, kwargs))    

How to create a function in python to be used from the terminal

Imagen
To write a python code able to read variables or inputs from the command line you only have to use sys.argv. Let's see a couple of examples. You can find the code here . File called example_v1.py: import sys def main(): print sys.argv if (__name__ == "__main__"): main(); Then you go to the folder where you have it and run this comman in the comman line: python example_v1.py Like here: As you can see the output is the name of the file. Let's try this: python example_v1.py hi hola ciao Basically the comand sys.argv is giving us a list of the commandline arguments received. Now I am going to create a program that is going to say hi and the name that you introduce after file name in the commandline. I called this script example_v2.py: import sys def main(): print 'Hi ' + sys.argv[1] + '!' if (__name__ == "__main__"): main(); So now we can write in the terminal: python

La broma de la estiba y como funcionan las cosas en España

Muchas gente en Inglatera me pregunta cómo puede ser que en España hayan esas tasas de paro en torno al 25% y la verdad es que es dificil de explicar. Sobre todo para gente que esta acostrumbrada al mercado de trabajo inglés. Para empezar el mercado de trabajo en inglaterra es muy flexible.  a) Hay variaciones de salarios atendiendo a las situaciones personales de cada uno, es decir, si uno es muy bueno en su trabajo podrá ganar más que otra persona que no lo he es tanto o no se esfuerza. Generalmente tu "manager" sabe muy lo que haces y lo que no, tu eficiencia e interes y tienes posibilidades de promocion y formacion en la misma empresa/universidad/escuela. Trabajes dónde trabajes hay gente. b) De la misma manera si la empresa no esta contenta contigo aunque no te despida no te volverá a contratar. A priori estas dos cosas pueden parece que son idénticas, pero el sentido de a) y b) es totalmente distinto en uno y otro. Por ejemplo conozco a muchos profes

Tunneling with Putty

Imagen
Tunneling with Putty Motivation I would like to create a ssh with a remote instance. Usually ssh is very secure, you can put password and key pair security. However it could be difficult to do other things in a more visual way, so we “tunnel” through the ssh connexion (taking advantage of it security). Here for a better explanation of Tunneling and ssh conection: https://chamibuddhika.wordpress.com/2012/03/21/ssh-tunnelling-explained/ Set up Preliminars: Of course the possibility of doing ssh is key. If you are going to “tunnel” using different port make sure those are open. You just have to issue your standard Putty session and add “X11 forwarding” as shown here: Also you should think which is going to be the port from your handy computer to the one you are connecting to. I want to use the port 8888 of the receiver and I am going to use port 9100 in the local machine: Make sure you press ADD, and open and/or save. Now you can connect: References

Cloudera with Openstack a Wordcount Example

Imagen
Cloudera with Openstack a Wordcount Example  Introduction 2 Set up the container 2 Run Mapreduce wordcount 3 Create Wordcount_mapper.py 3 Understanding the algorithm: 3 Create code: 3 Create Wordcount_reducer.py 4 Understanding the algorithm: 4 Create code: 5 Make those files executables 7 Create some data 7 Count the words. Run MapReduce 7 To check the results: 8 Changing Streaming options: 9 To see the results: 9 Try to notice the differences between the output when the reducers are run in Step 9, versus the output when there are no reducers and only the mapper is run in this step. The point of the task is to be aware of what the intermediate results look like. A successful submission will have words and counts that are not accumulated (which the reducer performs). Hopefully, this will help you get a sense of how data and tasks are split up in the map/reduce framework, and we will build upon that in the next lesson. 10 11. Change t