miércoles, 27 de enero de 2016

Web API (.NET): POSTing an object in XML format to an ApiController

Problem: In a .NET Web API app, inside a class which extends ApiController, it is desirable to receive an object in both XML and JSON format, in the most transparent form possible. Receiving the whole XML as a string, or using attributes (.NET's annotations) in the class to serialize are not acceptable solutions.

What tends to happen is that the object is always set to null when calling the method. There are various possible solutions, among them adding an "=" at the beginning of the POST message body. But doing so is not optimal, because it's a .NET specific convention and would limit us (if a partner used an HTTP client made with C++ running on Linux, we'd be in trouble). We cannot receive a string directly because that would prevent us from receiving the object in JSON format.

Solution: Receive the object with the attribute (.NET annotation) [FromBody], and enable the xml serialization inside our controller. An ideal place to do so would be the class' static constructor. For instance:

public class SomeController : ApiController{

    //Static constructor
    static SomeController(){
        //Needed for xml deserialization to work
        var xml = GlobalConfiguration.Configuration.Formatters.XmlFormatter;
        xml.UseXmlSerializer = true;
    }

    public async Task SomeMethodWhichReceivesBothXmlAndJson( [FromBody] SomeSerializableClass someInstance ){
        //...
    }

}

Web API (.NET): POSTeando un objeto en formato XML a un ApiController

Problema: En un aplicación .NET de tipo Web Api, dentro de una clase que hereda de ApiController, se desea poder recibir un objeto tanto en formato XML como JSON de la forma más transparente posible. Recibir el XML como string, o poner atributos (las annotations de .NET) en la clase a serializar no son soluciones aceptables.

Lo que suele ocurrir es que el objeto siempre llega como null. Hay varias soluciones posibles, entre ellas anteponer un "=" al principio del cuerpo del pedido POST. Pero eso no es aceptable, porque es algo específico de MVC y nos estaría limitando (si nuestro partner hace un cliente HTTP en C++ y Linux vamos a tener problemas). Tampoco podemos recibir un string porque eso nos impediría recibir JSON.

Solución: Recibir el objeto con el atributo (en el sentido de annotation de .NET) [FromBody], y habilitar el serializador xml en nuestro controller. Un lugar ideal para hacerlo es en el constructor estático de la clase. Por ejemplo:

public class SomeController : ApiController{

    //Static constructor
    static SomeController(){
        //Needed for xml deserialization to work
        var xml = GlobalConfiguration.Configuration.Formatters.XmlFormatter;
        xml.UseXmlSerializer = true;
    }

    public async Task SomeMethodWhichReceivesBothXmlAndJson( [FromBody] SomeSerializableClass someInstance ){
        //...
    }

}

sábado, 3 de octubre de 2015

Symfony 2 ignoring changes in parameters.yml

Say we want to change the database host because we changed our hosting service provider. We happily edit parameters.yml and upload it back up to the server, but we still get the "Connection timed out" error.

The solution is clearing the cache, but without direct access to the server, doing that via FTP might take ages. In my case, a simpler solution was to delete only the app/cache/prod/appProdProjectContainer.php file, which seems to be where Symfony 2 caches its parameters.

I'd like to personally thank the debug_print_backtrace() function. If you use it, make sure to do so with the DEBUG_BACKTRACE_IGNORE_ARGS flag, or you will get an illegible chunk of text.

Symfony 2 no toma cambios en parameters.yml

Supongamos que queremos cambiar el host de la base de datos porque migramos de hosting. Lo cambiamos alegremente en parameters.yml, pero sigue dando el error "Connection timed out".

La solución es borrar la cache, pero si no se tiene acceso directo al servidor, hacerlo por FTP puede demorar una eternidad. La solución en mi caso fue borrar sólo el archivo app/cache/prod/appProdProjectContainer.php, donde aparentemente Symfony 2 cachea los parámetros.

Quiero agradecer cordialmente a la función debug_print_backtrace() de PHP. Si la usan, asegúrense de invocarla con el flag DEBUG_BACKTRACE_IGNORE_ARGS, o les va a salir un choclo ilegible.

lunes, 14 de septiembre de 2015

How to handle a Win32 Unhandled Exception error in a Release build

You might have run into this scenario: a Windows application, which uses native/unmanaged C++, works perfectly fine in your development box. But in the productions environment, it crashes and a message like this one appears:

Sometimes, that exception can be caught with a try-catch block. But in other cases such as an Access violation, division by zero and similar ones, it can't be caught. Windows offers a specific solution for this: the __try - _except block, but it has some limitations. Specifically, it can only capture SEH exceptions. And those must be enabled on a project basis for this to work.

A better alternative, which is the one we'll analyze next, consists in using the SetUnhandledExceptionFilter function from the Kernel32.lib library. This allows us to catch the exception, but how can we get more information? Well, it is possible to obtain a stack trace by using additional functions from the psapi.lib and dbghlp.lib libraries. This way:

Usage example (StackTracing is a namespace, not a class):

void Init(){
   //...
   //Register handler
   ::SetUnhandledExceptionFilter(OnUnhandledException);
   //...
}


//Define handler (sorry for the Singleton, it's legacy code)
LONG WINAPI OnUnhandledException(  _In_  struct _EXCEPTION_POINTERS *pExceptionInfo ){
   string stackTrace;
   LONG result = StackTracing::DumpStackTrace( pExceptionInfo, stackTrace );
   ::GetErrorLoggerInstance()->Log( LOG_TYPE_ERROR, "StackTracing", "OnUnhandledException", stackTrace );
   ::MessageBoxA( NULL, "Fatal error encountered, application will shut down. Please check log for more information.", "DRA", MB_OK );
   return result;
}
This is the implementation of the helper functions used above:

//Get stack trace
#include 
#include 
#include 
#include 
#include 
#include 
#include 

#pragma comment(lib, "psapi.lib")
#pragma comment(lib, "dbghelp.lib")

// Some versions of imagehlp.dll lack the proper packing directives themselves
// so we need to do it.
#pragma pack( push, before_imagehlp, 8 )
#include 
#pragma pack( pop, before_imagehlp )

struct module_data {
    std::string image_name;
    std::string module_name;
    void *base_address;
    DWORD load_size;
};

DWORD DumpStackTrace( EXCEPTION_POINTERS *ep );
extern void YourMessage(const char* title, const char *fmt, ...);// Replace this with your own function

class symbol { 
    typedef IMAGEHLP_SYMBOL64 sym_type;
    sym_type *sym;
    static const int max_name_len = 1024;

public:
    symbol(HANDLE process, DWORD64 address) : sym((sym_type *)::operator new(sizeof(*sym) + max_name_len)) {
        memset(sym, '\0', sizeof(*sym) + max_name_len);
        sym->SizeOfStruct = sizeof(*sym);
        sym->MaxNameLength = max_name_len;
        DWORD64 displacement;

        if (!SymGetSymFromAddr64(process, address, &displacement, sym))
            throw(std::logic_error("Bad symbol"));
    }

    std::string name() { return std::string(sym->Name); }
    std::string undecorated_name() { 
        std::vector und_name(max_name_len);
        UnDecorateSymbolName(sym->Name, &und_name[0], max_name_len, UNDNAME_COMPLETE);
        return std::string(&und_name[0], strlen(&und_name[0]));
    }
};


class get_mod_info {
    HANDLE process;
    static const int buffer_length = 4096;
public:
    get_mod_info(HANDLE h) : process(h) {}

    module_data operator()(HMODULE module) { 
        module_data ret;
        char temp[buffer_length];
        MODULEINFO mi;

        GetModuleInformation(process, module, &mi, sizeof(mi));
        ret.base_address = mi.lpBaseOfDll;
        ret.load_size = mi.SizeOfImage;

        GetModuleFileNameExA(process, module, temp, sizeof(temp));
        ret.image_name = temp;
        GetModuleBaseNameA(process, module, temp, sizeof(temp));
        ret.module_name = temp;
        std::vector img(ret.image_name.begin(), ret.image_name.end());
        std::vector mod(ret.module_name.begin(), ret.module_name.end());
        SymLoadModule64(process, 0, &img[0], &mod[0], (DWORD64)ret.base_address, ret.load_size);
        return ret;
    }
};


// if you use C++ exception handling: install a translator function
// with set_se_translator(). In the context of that function (but *not*
// afterwards), you can either do your stack dump, or save the CONTEXT
// record as a local copy. Note that you must do the stack dump at the
// earliest opportunity, to avoid the interesting stack-frames being gone
// by the time you do the dump.
DWORD StackTracing::DumpStackTrace( EXCEPTION_POINTERS *ep, std::string& stackTrace ){
    HANDLE process = GetCurrentProcess();
    HANDLE hThread = GetCurrentThread();
    int frame_number=0;
    DWORD offset_from_symbol=0;
    IMAGEHLP_LINE64 line = {0};
    std::vector modules;
    DWORD cbNeeded;
    std::vector module_handles(1);

    // Load the symbols:
    if (!SymInitialize(process, NULL, false)) 
        throw(std::logic_error("Unable to initialize symbol handler"));
    DWORD symOptions = SymGetOptions();
    symOptions |= SYMOPT_LOAD_LINES | SYMOPT_UNDNAME;
    SymSetOptions(symOptions);
    EnumProcessModules(process, &module_handles[0], module_handles.size() * sizeof(HMODULE), &cbNeeded);
    module_handles.resize(cbNeeded/sizeof(HMODULE));
    EnumProcessModules(process, &module_handles[0], module_handles.size() * sizeof(HMODULE), &cbNeeded);
    std::transform(module_handles.begin(), module_handles.end(), std::back_inserter(modules), get_mod_info(process));
    void *base = modules[0].base_address;

    // Setup stuff:
    CONTEXT* context = ep->ContextRecord;
#ifdef _M_X64
    STACKFRAME64 frame;
    frame.AddrPC.Offset = context->Rip;
    frame.AddrPC.Mode = AddrModeFlat;
    frame.AddrStack.Offset = context->Rsp;
    frame.AddrStack.Mode = AddrModeFlat;
    frame.AddrFrame.Offset = context->Rbp;
    frame.AddrFrame.Mode = AddrModeFlat;
#else
    STACKFRAME64 frame;
    frame.AddrPC.Offset = context->Eip;
    frame.AddrPC.Mode = AddrModeFlat;
    frame.AddrStack.Offset = context->Esp;
    frame.AddrStack.Mode = AddrModeFlat;
    frame.AddrFrame.Offset = context->Ebp;
    frame.AddrFrame.Mode = AddrModeFlat;
#endif
    line.SizeOfStruct = sizeof line;
    IMAGE_NT_HEADERS *h = ImageNtHeader(base);
    DWORD image_type = h->FileHeader.Machine;
    int n = 0;

    // Build the string:
    std::ostringstream builder;
    do {
        if ( frame.AddrPC.Offset != 0 ) {
            std::string fnName = symbol(process, frame.AddrPC.Offset).undecorated_name();
            builder << fnName;
            if (SymGetLineFromAddr64( process, frame.AddrPC.Offset, &offset_from_symbol, &line)) 
                builder << "  " /*<< line.FileName*/ << "(" << line.LineNumber << ")\n";
            else builder << "\n";
            if (fnName == "main")
                break;
            if (fnName == "RaiseException") {
                // This is what we get when compiled in Release mode:
                //YourMessage("Crash", "Your program has crashed.\n\n");
                return EXCEPTION_EXECUTE_HANDLER;
            }
        }
        else
            builder << "(No Symbols: PC == 0)";
        if (!StackWalk64(image_type, process, hThread, &frame, context, NULL, 
                            SymFunctionTableAccess64, SymGetModuleBase64, NULL))
            break;
        if (++n > 10)
            break;
    } while (frame.AddrReturn.Offset != 0);
    //return EXCEPTION_EXECUTE_HANDLER;
    SymCleanup(process);

    // Display the string:
    //::MessageBoxA( NULL, builder.str().c_str(), "Stack Trace", MB_OK );
    stackTrace = builder.str();
    return EXCEPTION_EXECUTE_HANDLER;
}

Cómo manejar Win32 Unhandled Exception en Release

Probablemente se hayan topado con este escenario: una aplicación Windows, que usa C++ nativo/no manejado, funciona perfecto en la máquina de desarrollo. Pero en las máquinas de los clientes a veces sale un message box similar al siguiente:

A veces, esa excepción puede capturarse con un bloque try-catch. Pero en ciertos casos como Access violation, división por cero y similares, no se puede capturar. Una alternativa específica de Windows es usar un bloque __try - __except, pero tiene sus limitaciones. Concretamente, sólo puede capturar excepciones de tipo SEH. Y las mismas deben estar habilitadas a nivel proyecto.

Una mejor alternativa, que es la que estudiaremos a continuación, consiste en usar la función SetUnhandledExceptionFilter de la biblioteca Kernel32.lib. Esto nos permite capturar la excepción, ¿pero cómo obtener más información? Es posible obtener un stack trace usando unas funciones auxiliares de las bibliotecas psapi.lib y dbghlp.lib. De este modo:

Ejemplo de uso (StackTracing es un namespace, no una clase):

void Init(){
   //...
   //Registro handler
   ::SetUnhandledExceptionFilter(OnUnhandledException);
   //...
}

//Defino handler (disculpen el Singleton, es código legacy)
LONG WINAPI OnUnhandledException(  _In_  struct _EXCEPTION_POINTERS *pExceptionInfo ){
   string stackTrace;
   LONG result = StackTracing::DumpStackTrace( pExceptionInfo, stackTrace );
   ::GetErrorLoggerInstance()->Log( LOG_TYPE_ERROR, "StackTracing", "OnUnhandledException", stackTrace );
   ::MessageBoxA( NULL, "Fatal error encountered, application will shut down. Please check log for more information.", "DRA", MB_OK );
   return result;
}
Implementación de las funciones definidas arriba:

//Obtengo stack trace
#include 
#include 
#include 
#include 
#include 
#include 
#include 

#pragma comment(lib, "psapi.lib")
#pragma comment(lib, "dbghelp.lib")

// Some versions of imagehlp.dll lack the proper packing directives themselves
// so we need to do it.
#pragma pack( push, before_imagehlp, 8 )
#include 
#pragma pack( pop, before_imagehlp )

struct module_data {
    std::string image_name;
    std::string module_name;
    void *base_address;
    DWORD load_size;
};

DWORD DumpStackTrace( EXCEPTION_POINTERS *ep );
extern void YourMessage(const char* title, const char *fmt, ...);// Replace this with your own function

class symbol { 
    typedef IMAGEHLP_SYMBOL64 sym_type;
    sym_type *sym;
    static const int max_name_len = 1024;

public:
    symbol(HANDLE process, DWORD64 address) : sym((sym_type *)::operator new(sizeof(*sym) + max_name_len)) {
        memset(sym, '\0', sizeof(*sym) + max_name_len);
        sym->SizeOfStruct = sizeof(*sym);
        sym->MaxNameLength = max_name_len;
        DWORD64 displacement;

        if (!SymGetSymFromAddr64(process, address, &displacement, sym))
            throw(std::logic_error("Bad symbol"));
    }

    std::string name() { return std::string(sym->Name); }
    std::string undecorated_name() { 
        std::vector und_name(max_name_len);
        UnDecorateSymbolName(sym->Name, &und_name[0], max_name_len, UNDNAME_COMPLETE);
        return std::string(&und_name[0], strlen(&und_name[0]));
    }
};


class get_mod_info {
    HANDLE process;
    static const int buffer_length = 4096;
public:
    get_mod_info(HANDLE h) : process(h) {}

    module_data operator()(HMODULE module) { 
        module_data ret;
        char temp[buffer_length];
        MODULEINFO mi;

        GetModuleInformation(process, module, &mi, sizeof(mi));
        ret.base_address = mi.lpBaseOfDll;
        ret.load_size = mi.SizeOfImage;

        GetModuleFileNameExA(process, module, temp, sizeof(temp));
        ret.image_name = temp;
        GetModuleBaseNameA(process, module, temp, sizeof(temp));
        ret.module_name = temp;
        std::vector img(ret.image_name.begin(), ret.image_name.end());
        std::vector mod(ret.module_name.begin(), ret.module_name.end());
        SymLoadModule64(process, 0, &img[0], &mod[0], (DWORD64)ret.base_address, ret.load_size);
        return ret;
    }
};


// if you use C++ exception handling: install a translator function
// with set_se_translator(). In the context of that function (but *not*
// afterwards), you can either do your stack dump, or save the CONTEXT
// record as a local copy. Note that you must do the stack dump at the
// earliest opportunity, to avoid the interesting stack-frames being gone
// by the time you do the dump.
DWORD StackTracing::DumpStackTrace( EXCEPTION_POINTERS *ep, std::string& stackTrace ){
    HANDLE process = GetCurrentProcess();
    HANDLE hThread = GetCurrentThread();
    int frame_number=0;
    DWORD offset_from_symbol=0;
    IMAGEHLP_LINE64 line = {0};
    std::vector modules;
    DWORD cbNeeded;
    std::vector module_handles(1);

    // Load the symbols:
    if (!SymInitialize(process, NULL, false)) 
        throw(std::logic_error("Unable to initialize symbol handler"));
    DWORD symOptions = SymGetOptions();
    symOptions |= SYMOPT_LOAD_LINES | SYMOPT_UNDNAME;
    SymSetOptions(symOptions);
    EnumProcessModules(process, &module_handles[0], module_handles.size() * sizeof(HMODULE), &cbNeeded);
    module_handles.resize(cbNeeded/sizeof(HMODULE));
    EnumProcessModules(process, &module_handles[0], module_handles.size() * sizeof(HMODULE), &cbNeeded);
    std::transform(module_handles.begin(), module_handles.end(), std::back_inserter(modules), get_mod_info(process));
    void *base = modules[0].base_address;

    // Setup stuff:
    CONTEXT* context = ep->ContextRecord;
#ifdef _M_X64
    STACKFRAME64 frame;
    frame.AddrPC.Offset = context->Rip;
    frame.AddrPC.Mode = AddrModeFlat;
    frame.AddrStack.Offset = context->Rsp;
    frame.AddrStack.Mode = AddrModeFlat;
    frame.AddrFrame.Offset = context->Rbp;
    frame.AddrFrame.Mode = AddrModeFlat;
#else
    STACKFRAME64 frame;
    frame.AddrPC.Offset = context->Eip;
    frame.AddrPC.Mode = AddrModeFlat;
    frame.AddrStack.Offset = context->Esp;
    frame.AddrStack.Mode = AddrModeFlat;
    frame.AddrFrame.Offset = context->Ebp;
    frame.AddrFrame.Mode = AddrModeFlat;
#endif
    line.SizeOfStruct = sizeof line;
    IMAGE_NT_HEADERS *h = ImageNtHeader(base);
    DWORD image_type = h->FileHeader.Machine;
    int n = 0;

    // Build the string:
    std::ostringstream builder;
    do {
        if ( frame.AddrPC.Offset != 0 ) {
            std::string fnName = symbol(process, frame.AddrPC.Offset).undecorated_name();
            builder << fnName;
            if (SymGetLineFromAddr64( process, frame.AddrPC.Offset, &offset_from_symbol, &line)) 
                builder << "  " /*<< line.FileName*/ << "(" << line.LineNumber << ")\n";
            else builder << "\n";
            if (fnName == "main")
                break;
            if (fnName == "RaiseException") {
                // This is what we get when compiled in Release mode:
                //YourMessage("Crash", "Your program has crashed.\n\n");
                return EXCEPTION_EXECUTE_HANDLER;
            }
        }
        else
            builder << "(No Symbols: PC == 0)";
        if (!StackWalk64(image_type, process, hThread, &frame, context, NULL, 
                            SymFunctionTableAccess64, SymGetModuleBase64, NULL))
            break;
        if (++n > 10)
            break;
    } while (frame.AddrReturn.Offset != 0);
    //return EXCEPTION_EXECUTE_HANDLER;
    SymCleanup(process);

    // Display the string:
    //::MessageBoxA( NULL, builder.str().c_str(), "Stack Trace", MB_OK );
    stackTrace = builder.str();
    return EXCEPTION_EXECUTE_HANDLER;
}

jueves, 10 de septiembre de 2015

Toptal

Today, I was browsing Facebook on the bus to college and I stumbled upon an article about Toptal. To summarize, Toptal is a company created by two enginnering students. What sets it apart is the fact that nowadays it made profits for 80 million dollars, rejected Silicon Valley offers and... never had an office.

All interactions, meeting and initial recruitment was done via Skype. The creators designed a special screening process aimed at recruiting "the top 3% best" engineers across the world. The whole process is done remotely, and includes tests and Skype meetings.

I have worked professionally as a programmer since 2009, and recently started getting into the freelancing world. Some months ago, I created a profile on both Freelancer and Upwork, and managed to win, complete and get paid for a few projects. As we all know:

  • It's great not having to commute to office everyday!
  • Having the freedom to choose which work to do is a great perk: I can choose those I like and can do in the most efficient way. It's a win/win situation.
  • I gained the extra time I desperately need to finish my Computer Engineering carreer.
  • The constant commuting between work, home and college was slowly wearing me out. The day was not long enough to get things done, and I was wasting my energy

However, freelancing has its downsides:

  • Unstability: you might win all bids one month, and have a lousy streak the next. This is specially true when you are getting started and don't have high reputation.
  • Low growth rate: the strong competition forces you to lowball when defining your pay, and the initial earnings are too low.
So I'm going to give a shot... try getting into that top 3%...