Saturday, 28 October 2017

C++ 2017

Wahooo!!! C++ 17 has now been published as an ISO standard!!

I haven't used C++ for a long time as C# is my daily driver these days. But I will always remember when I moved from C++ to C#, being so much more productive as the language was a more safer environment and there was less things you had to worry about.

However, I always try to keep up-to-date with news over in the C++ camp. The C++ language will always have a special place in my heart and I have plenty of war stories and battle scars!. Yes, C++ is an overly complicated language with years of baggage but with every new feature that is added there is less and less sharp bits to impale yourself on.

The only thing I would like the standards committee to do is actually remove old/legacy features from the language. And maybe provide a compiler switch to enable them for backwards compatibility. This would mean you have to opt-in to use those legacy features. Unfortunately I don't think this will ever happen.

Of course, there was a time when C++ languished but that all changed in 2011 with C++11. It added so many features to the language like lambdas and auto, unique pointer and shared pointer that it changed the way you code. For example, now, when coding C++ they say if you're writing "new" or "delete" you're doing it wrong! Instead you should be using make_shared() or make_unique() which means you don't have to worry about memory leaks (as much).

C++ 20

What piqued my interest is what is coming down the pipe for C++20 and beyond. Big things are brewing in the C++ world and C++ 20 is where all the big action is. It's 3 years away but it promises:

Okay, so I kinda snuck metaclasses in that list. It might be a little too early for them to make the cut for C++20. But a guy can dream can't he?

The biggest problem I have with C++ at the moment is the #include header system. It's soo old and antiquated. Coming from Java or C# where they have a module system the #include system is painful. But hopefully C++20 will fix that with it's new module system (And then maybe we can get an official package manager).

C++ Core Guidelines

However, the thing about C++ is it's as "old as god's dog" which means when searching on the internet, you need to make sure you are reading about the latest stuff. While there maybe less sharp pointy bits, for the most-part the old stuff is still there and you need to know what to avoid!

You don’t want to be reading old out-of-date information. Thankfully the C++ guys (Bjarne & Herb) are working on the C++ Core Guidelines. Apparently, these Guidelines are a "set of rules designed to help you write modern, safe C++ – saving you time and effort as well as making your code more reliable." Microsoft even have a nuget package add-in for Visual Studio that performs code analysis to check your code for compliance!

Wrapping Up

In a world where we have modern, new and shiny languages like Rust, Swift & Kotlin you'd be forgiven for thinking there is no place for C++. That it's time to retire the old dog and put her out to pasture (tounge == in-cheek). Of course, we know that's not the case when we are talking about a language as important as C++. It's just good to see that C++ is alive and well and I am watching with interest to see how the language continues to evolve.


Contact Me:  ocean.airdrop@gmail.com

Friday, 27 October 2017

INotifyPropertyChanged & Fody

We all know what the INotifyPropertyChanged interface does. It can be used to raise an event when a property of a class changes. Then in another section of code we can subscribe to these events and perform certain actions based on the application needs. It's all very cool but also old hat and pedestrian.

But the thing with this interface is that you can end up writing a lot of boiler plate code! For example, take this simple person class:

public class Person
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public DateTime DateOfBirth { get; set; }
}

It's nice. It's neat.. All is good with the world. But then you decide you want to be notified when one of the fields in the class is changed, so we introduce the INotifyPropertyChanged interface. Suddenly, for every property we find ourselves expanding the above code like this:

private string m_firstName;
public string FirstName
{
    get
    {
        return m_firstName;
    }
    set
    {
        OnPropertyChanged("FirstName", m_firstName, value);
        m_firstName = value;
    }
}

For every property you need to include a backing field, then fill in the getter and setter functions yourself and in the setter field ensure you raise the property changed event handler.. Geez! If you need to add this to a number of objects within a sizeable project, the work can quickly become monotonous.

The full class definition now looks like this:

class PersonNotify : INotifyPropertyChanged
{
    private string m_firstName;
    public string FirstName
    {
        get { return m_firstName; }
        set
        {
            OnPropertyChanged("FirstName", m_firstName, value);
            m_firstName = value;
        }
    }

    private string m_lastName;
    public string LastName
    {
        get { return m_lastName; }
        set
        {
            OnPropertyChanged("LastName", m_lastName, value);
            m_lastName = value;
        }
    }

    private DateTime m_dateOfBirth;
    public DateTime DateOfBirth
    {
        get { return m_dateOfBirth; }
        set
        {
            OnPropertyChanged("DateOfBirth", m_dateOfBirth, value);
            m_dateOfBirth = value;
        }
    }

    public event PropertyChangedEventHandler PropertyChanged;

    public void OnPropertyChanged(string propertyName, object before, object after)
    {
        if (PropertyChanged != null)
            PropertyChanged.Invoke(this, new PropertyChangedEventArgs(propertyName));
    }
}

This is the situation I was in when I thought to myself “surely there must be a better way?”. And, in the internet age, if you can think of it, chances are someone else has already implemented it!

Enter Fody! You can find the nuget package here and install it with nuget like this:

Install-Package PropertyChanged.Fody 

It’s a great little utility which, at compile time, looks for classes that implement the INotifyPropertyChanged interface and implements the backing fields for each property of your class as well as raising the event for you.

With fody installed we can now revert back to our original class with just a couple of small modifications:

public class Person : INotifyPropertyChanged
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public DateTime DateOfBirth { get; set; }

    public event PropertyChangedEventHandler PropertyChanged;

    public void OnPropertyChanged(string propertyName, object before, object after)
    {
        if (PropertyChanged != null)
            PropertyChanged.Invoke(this, new PropertyChangedEventArgs(propertyName));
    }
}

In the code above we have implemented the INotifyPropertyChanged interface and added a small bit of code to raise an event when any of the properties change.

Side note: In my final code I also extract the event and OnPropertyChanged handler into a central base class which cleans up all the model classes that derive from it.

Now, when you build it you will see this output in the Visual Studio build window:

1>------ Build started: Project: FodyTest, Configuration: Debug Any CPU ------
1>    Fody: Fody (version 2.0.0.0) Executing
1>      Fody/PropertyChanged:    No reference to 'PropertyChanged' found. References not modified.
1>    Fody:   Finished Fody 52ms.
1>    Fody:   Skipped Verifying assembly since it is disabled in configuration
1>    Fody:   Finished verification in 0ms.
1>  FodyTest -> C:\OceanAirdrop\TempCode\FodyTest\FodyTest\bin\Debug\FodyTest.exe
========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========

Now that’s nice and saves us a lot of work. Here's the full implementation:

class Program
{
    static void Main(string[] args)
    {
        var p = new Person();
        p.PropertyChanged += PropertyChangedEvent;
        p.FirstName = "Berty";
        p.LastName = "Burnstein";
        p.DateOfBirth = DateTime.Now;
    }

    private static void PropertyChangedEvent(object sender, PropertyChangedEventArgs e)
    {
        var propertyChanged = (OceanAirdropPropertyChangedArgs)e;
        Trace.WriteLine(string.Format("{0} changed from {1} to {2}.", 
        propertyChanged.PropertyName, propertyChanged.Before, propertyChanged.After));
    }
}

// Create your model objects as normal, but derive from BaseData and INotifyPropertyChanged
public class Person : BaseData, INotifyPropertyChanged
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public DateTime DateOfBirth { get; set; }
}

// Create a base class that implements the INotifyPropertyChanged interface and raises the event
public class BaseData : INotifyPropertyChanged
{
    public event PropertyChangedEventHandler PropertyChanged;

    public void OnPropertyChanged(string propertyName, object before, object after)
    {
        if (PropertyChanged != null)
            PropertyChanged.Invoke(this, new OceanAirdropPropertyChangedArgs(propertyName, before, after));
    }
}

// Create own event args that capture property value before and after!
public class OceanAirdropPropertyChangedArgs : PropertyChangedEventArgs
{
    public OceanAirdropPropertyChangedArgs(string propertyName) : base(propertyName) { }

    public OceanAirdropPropertyChangedArgs(string propertyName, object before, object after) : base(propertyName)
    {
        Before = before; After = after;
    }
    public virtual object Before { get; }
    public virtual object After { get; }
}

So there we have it. I have written this blog post as a reminder for me, the next time I need to implement notification changes in another project. Fody is a nice little utility to be aware of and kept in our software toolbox.


Contact Me:  ocean.airdrop@gmail.com

Thursday, 31 August 2017

Professional Winform Controls & Libraries

Yes, yes, I know what you’re thinking. Winforms development! But… Erm… Isn’t Winforms dead?

I wouldn’t say Winforms is dead - let’s just say it’s “been done”! It’s complete. Whilst the Winforms framework is not as hot or current as it once was, it’s still very capable for a lot of business tasks and needs. And it’s still my go-to choice for developing in-house “line of business apps” that have no need to be online.

The fact of the matter is, there are some apps which don’t need to be put on a mobile phone or be accessible from the web. The desktop isn’t going away anytime soon and traditional desktop applications still have their place. These line of business apps can be considered as "dark matter" apps which keeps the cogs of businesses turning. The majority of people will never see them but they are out there being used daily.

Good software solves business problems no matter what form it comes in. As developers, we should always be looking for the minimum viable product. What’s the least amount of code we need to write to get the job done to solve the business needs? The majority of time a simple traditional desktop application will do the trick.

As an aside, I also make use of ClickOnce which gives you all the deployment benefits of a Web App inside a company. You publish an update and the next time your users load the app they are on the latest code.

But just because Winforms is old technology doesn’t mean the applications can’t be fancy-pants-looking! Because this area is so well-trodden there is a rich ecosystem of free controls out there which you can use to spice up your app. These apps don’t have to be boring battleship grey! You’re not stuck with the controls you get by default in Visual Studio.

With that said, here’s a rundown of some controls I have used or come across which add shine to any app:

Krypton .NET WinForms controls

"The Krypton Suite of .NET WinForms controls are now freely available for use in personal or commercial projects." That’s a quote straight from the GitHub pages of ComponentFactory.

This is a great library and includes a ribbon control, docking controls, an enhanced tab control (which they call a navigator). This is definitely one to check out!

Check it out here: https://github.com/ComponentFactory/Krypton

Syncfusion .NET WinForms controls

Syncfusion now has a community license and is available to all!

“The community license is our way of giving back to the community,” says Daniel Jebaraj, vice president of Syncfusion. “We want to support the individual and small business developer by offering our tremendous capabilities at no cost, and with no expiration date. What we offer is not a subset of Essential Studio but the real deal.

Customers using the community license will receive the exact same bits that we ship to our other customers. What is more exciting is that we offer free technical support to every customer licensed under the community license!”

Syncfusion has a whole slew of controls but of interest to me is their Data Grid and Spreadsheets controls with their excel like UI.

Check it out here: https://www.syncfusion.com/products/communitylicense

HTML Renderer

This is a fantastic little HTML framework. It’s a lightweight HTML Rendering library which means you can embed any HTML UI elements in any control you want and expand them beyond their original implementation. For example, I have previously used this to embed HTML in a DataGridView Cell.

Check it out here: https://github.com/ArthurHub/HTML-Renderer


Advanced DataGridView

I use this control all the time. It provides excel like filtering over multiple columns.

Check it out here: https://github.com/davidegironi/advanceddatagridview

ObjectListView

This is a flexible replacement for the built in ListView control. It’s feature rich and has animations, filtering, drag-drop, and even a tree list version.

Check it out here: http://objectlistview.sourceforge.net/cs/index.html

Microsoft Chart Controls

Need to display a chart? Microsoft's own chart controls have a large selection to choose from with great documentation

Check it out here: https://www.microsoft.com/en-gb/download/details.aspx?id=14422

CefSharp - That’s Chrome all up in your Winforms!

CefSharp enables you to bundle the open source Chromium web browser in your application, with the added ability to execute code in JavaScript land from C# and vice-versa. This tool is powerful!

Check it out here: https://github.com/cefsharp/CefSharp

Excel EPPlus

If you’re writing a line of business app, sooner or later you’re going to come into contact with Excel. This library is your friend. It’s quite simply awesome. It allows you to generate advanced excel spreadsheets from a C# application.

Check it out here: http://epplus.codeplex.com/

Conclusion

There you have it. While Winforms might no longer be classed as one of the cool kids, you can still be very productive in it and these frameworks imbue you with the power to create good looking applications.


Contact Me:  ocean.airdrop@gmail.com

Sunday, 5 March 2017

Programmatic Analysis of Wireshark Log Files using C#

The other day, I wanted to perform some Wireshark filtering on a .pcap file to obtain a count of the packets found for a large number of IP addresses.

I wanted to find out the number of tcp retransmissions for a specified IP address, as well as the count of TCP resets for each IP address. And finally, I wanted to get a count of the number of "keep alive" packets for each IP address.

Okay, so this is pretty easy to perform in Wireshark. Just filter the traffic with the following filters:

tcp.analysis.retransmission && ip.addr == 1.2.3.4
tcp.flags.reset == 1 && ip.addr == 1.2.3.4
tcp.analysis.keep_alive && ip.addr == 1.2.3.4  

But I didn't want to go through the user interface for hundreds of different IP addresses. I wanted to do this programatically, in code.

Now, there are a couple of different approaches you can take here depending on your requirements.

At first I used PcapDotNet. This is a great library and you can walk the packets in the file and explore the individual properties of a packet.

Simply download the binaries from here. Then reference them in your project and you're off!

The code to get up and running is simple. The code below uses the function IncomingPacketHandler to walk every packet in the .pcap file:

class Program
{
   static int m_packetNumber = 0;
     
   static void Main(string[] args)
   {
      string file = @"C:\WireSharkAnalysis\capture2.pcap";
      // Create the offline device
      OfflinePacketDevice selectedDevice = new OfflinePacketDevice(file);
         
      // 65536 guarantees that the whole packet will be captured on all the link layers
      int readWholePacket = 65536;
      
      // read timeout
      int readTimeOut = 1000;
     
      using ( PacketCommunicator communicator = selectedDevice.Open( readWholePacket, PacketDeviceOpenAttributes.Promiscuous, readTimeOut))
      {
         communicator.ReceivePackets(0, IncommingPacketHandler);
      }
   }

    private static void IncommingPacketHandler(Packet packet)
    {
        // This function will get called for every packet in the .pcap file!
        m_packetNumber++;

        Console.WriteLine( packet.Timestamp.ToString( "yyyy-MM-dd hh:mm:ss.fff") + " length:" + packet.Length);

        var testIP = new IpV4Address("10.1.1.1");

        if (packet.Ethernet.IpV4.Tcp.IsReset == true )
        {
            // do something  
        }

        if (packet.Ethernet.IpV4.Tcp.ControlBits.HasFlag( PcapDotNet.Packets.Transport.TcpControlBits.Acknowledgment) == true &&
            packet.Ethernet.IpV4.Tcp.ControlBits.HasFlag( PcapDotNet.Packets.Transport.TcpControlBits.Push) == true )
        {
            // do something       
        }

        if (packet.Ethernet.IpV4.Tcp.IsReset == false)
        {
            // do something  
        }

        if (packet.Ethernet.IpV4.Source == testIP)
        {
            // do something  
        }

        Console.WriteLine();
    }
}

You can perform deep inspection of the packet as seen below in the "quick watch" window. In short, you get access to everything.

Very neat!

But here's the thing - I wanted to get a count of the number of tcp retransmissions. That information is not available as part of each individual packet. Apparently Wireshark "compares the sequence numbers to what it has determined to be the next expected sequence number" to allow you to filter them.

This is easy in Wireshark. Just type "tcp.analysis.retransmission" into the filter bar and it will display the TCP Retransmissions. The filter is part of the TCP analysis that Wireshark performs when reading the packets.

So, that's when I turned to my second option: Using tshark.exe (the command line version of Wireshark) to read in a file and pass my filter to. I wrapped the tshark command line tool in a simple class, but the main work-horse is this function here:

public int ProcessFilter(string filter)
{
    // Stage 1: Setup the wireshark filter command
    m_tsharkCmd = string.Format(m_tsharkTemplate, m_tsharkPath, m_pcapFile, filter);

    // Stage 2: Clear the output
    m_tsharkOutput.Clear();

    // Stage 3: Run the command!
    using (m_process = new Process())
    {
        m_process.StartInfo.WorkingDirectory = @"C:\";
        m_process.StartInfo.FileName = Path.Combine(Environment.SystemDirectory, "cmd.exe");
        m_process.StartInfo.UseShellExecute = false;
        m_process.StartInfo.RedirectStandardInput = true;
        m_process.StartInfo.RedirectStandardOutput = true;
        m_process.StartInfo.RedirectStandardError = true;
        m_process.OutputDataReceived += OutputHandler;
        m_process.ErrorDataReceived += OutputHandler;
        m_process.Exited += new EventHandler(process_Exited);
        m_process.Start();
        m_process.BeginOutputReadLine();
        m_process.BeginErrorReadLine();

        // Send a directory command and an exit command to the shell
        m_process.StandardInput.WriteLine(m_tsharkCmd);
        m_process.StandardInput.WriteLine("exit");
        m_process.WaitForExit();
        m_process.Close();               
    }

    // Stage 4: Output the number of packets!
    int packetCount = GetPacketCount();
    return packetCount;
}

It's a quick and dirty approach but hey, it works!

Using this approach, means I can loop around on a number of different IP addresses and issue the previous Wireshark filters I had above to find the number of tcp packet retransmissions.

tcp.analysis.retransmission && ip.addr == 1.2.3.4
tcp.flags.reset == 1 && ip.addr == 1.2.3.4
tcp.analysis.keep_alive && ip.addr == 1.2.3.4  

This small project can be found on my GitHub pages here. It's a simple stub project but can easily be expanded to perform different types analysis on any .pcap file.


Contact Me:  ocean.airdrop@gmail.com

Saturday, 18 February 2017

Using Redis as a Data Caching Server

Everyone knows how important caching is in computing. We are surrounded by caching managers here, there and everywhere. You've got your CPU caching in the form of L1 and L2 cache which stores the next bit of data the CPU needs. GPUs have a cache. All hard drives come equipped with an on-board cache. Database servers heavily cache your most used queries and query plans, web servers cache the most used data and web clients (browsers) cache client side data.

Basically, caching is everywhere.

Why am I mentioning this? Well, recently I have been redesigning a framework that plans to make use of cached data to save trips to the database and Redis looks very enticing.

But what's the cost/impact of not using a cache?

Well, I found this great link "Latency Numbers Every Programmer Should Know", which orders the latency numbers of accessing the "CPU cache" all the way up to connecting to a computer over the open internet.

The numbers shouldn't surprise you. Basically accessing memory is wicked fast! (It can be referenced in 100ns). If you have the data in memory versus retrieving it from the database on another server it's a no-brainer! - Memory wins every time.

That's where Redis comes in. - It's a fast in-memory data cache/NoSQL database.

It's an open source server software and they have a client for every programming language imaginable. It's also used by websites like Twitter, GitHub, Stack Overflow, etc.

Installation (on Windows)

The download pages of Redis explain that the "Microsoft Open Tech group" support the windows port of Redis. It's available to download via nuget here. Just: "Install-Package Redis-64"

Incidently, it looks like Azure allows you to setup a Redis cache on their service. I wonder if they are using the same version of Redis they are hosting on nuget?!

When you run the server from the command line you will see the Redis logo and that the server is accepting connections on port 6379 (the default port).

This runs the Redis server in interactive mode.

To install Redis as a windows service run this command redis-server --service-install redis.windows.conf --loglevel verbose. To uninstall the service run this command: redis-server --service-uninstall

You can then go and start Redis as a windows service

That's it! The server is installed!

ServiceStack.Redis Client

Once you have got the server installed, the next thing you will need to do is download a client. The best client for C# is ServiceStack.Redis

So what's the code look like?

Well, it's nice and simple. Just what we like. For example the following code adds some data to the cache, then retrieves it later on.

using ServiceStack.Redis;

class Program
{
    static void Main(string[] args)
    {
        string someOrder = "some data to cache";

        // Store some Data
        using (RedisClient client = new RedisClient("127.0.0.1", 6379))
        {
            client.SetValue("order:1", someOrder);
        }

        // Later on! - Lets get some data from the cache.
             
        using (IRedisClient client = new RedisClient("127.0.0.1", 6379))
        {
            var result = client.GetValue("order:1");
        }
    }
}

Expiring Data

As we all know, one important aspect of a cache is knowing when to let go of the cached data. This can give you headaches if you hold on to data too long. This is application specific and you will need to understand the time limits for when certain types of data goes stale.

Here's a more complicated example that serialises a class as well as expiring the data after 10 seconds.

using ServiceStack.Redis;
class Person
{
    public string Name { get; set; }
    public int Age { get; set; }

    public Person(string n, int a)
    {
        Name = n; Age = a;
    }
}

class Program
{
    static void Main(string[] args)
    {
        Person person1 = new Person("Rick", 45);
        Person person2 = new Person("Morty", 18);

        string jsonString = "";

        int hours = 0, mins = 0, secs = 10;
        TimeSpan expireIn = new TimeSpan(hours, mins, secs);

        // Store some Data
        using (RedisClient client = new RedisClient("127.0.0.1", 6379))
        {
            jsonString = Newtonsoft.Json.JsonConvert.SerializeObject(person1);
            client.SetValue("person:1", jsonString, expireIn);

            jsonString = Newtonsoft.Json.JsonConvert.SerializeObject(person2);
            client.SetValue("person:2", jsonString, expireIn);
        }

        // Later on! - Lets get some data from the cache.
             
        using (IRedisClient client = new RedisClient("127.0.0.1", 6379))
        {
            var result = client.GetValue("person:2");
            
            if ( result != null)
            {
                var pTemp = Newtonsoft.Json.JsonConvert.DeserializeObject(result);
            }
        }

        // Much Later on (after 10 secs)! - Lets get some data from the cache.
        Thread.Sleep(1000 * 12);

        using (IRedisClient client = new RedisClient("127.0.0.1", 6379))
        {
            var result = client.GetValue("person:2");

            // The result will be null because more than 10 secs have elapsed
            if (result != null)
            {
                var pTemp = Newtonsoft.Json.JsonConvert.DeserializeObject(result);
            }
        }
    }
}

Wrapping Up!

There's a lot more to Redis than I have mentioned here. It has a full suite of commands and can be used in many other ways apart from a cache - but it makes a great cache manager!


Contact Me:  ocean.airdrop@gmail.com

Sunday, 12 February 2017

Code and Music

Do you listen to music when you code?

I do.

For me, there is no better experience of falling into a code-hole while being wisked away by a backing track of your liking.

I find the two compliment each other.

If you have a sufficiently meaty problem and an inkling of how you're going to solve it, music can sweep away time.

You tumble down that code-hole and before you know it, you look up and a couple of hours have passed.

Something happens in our brains, where your logic and reasoning about a problem develops to the point where you get a nice flow going. It's hard to explain but it happens.

I am in no way precious about what type of music I listen to. In fact I listen to a lot of different genres but I find nothing compliments coding better than a DJ set.

I love music (who doesn't). I collect music. I grew up being fed and watered on all types of dance music genres and back in the day, listening to the BBC Essential Mix was a staple! - In today's landscape music has never been more accessible with podcasts, youtube, soundcloud, mixcloud etc. There is so much fuel out there to use to code to.

My takeaway? Let the music play, and don't forget to turn it up to 11.


Contact Me:  ocean.airdrop@gmail.com

Sunday, 5 February 2017

SeriLog & Application Diagnostic Logging

Show me an application that doesn't log anything and i'll show you an application which is hard to debug.

Every application we write should have a healthy sprinkling of diagnostic logging embedded throughout its logic. A trace to show us what's happening on the inside from the outside. In the past, I've hand-rolled my own bespoke logging classes to output either to file, database or simply console.

Not any more!

The problem has been solved. It's been done. There are libraries out there like Log4Net or Nlog

But from now on, I intend to standardise all my C# code to use SeriLog.

SeriLog is an example of a library where so many developers have poured in so much time that any home grown library cannot compete.

Why waste your time? Just "nuget" it into your project and be done. Move on.

Why is it so cool?

Because it's simple. And because it supports many, many outputs. They call them sinks and there is a sink for everything as you can see in the image above.

But the real cool part is that you can pass a data model to SeriLog and it will serialise the properties in JSON format.

Want to see some sample code? The github pages have plenty of samples there but the code sample below demonstrates how SeriLog logs to:

  • The screen
  • A rolling log file on disk
  • And a database table
using Serilog;

namespace OceanAirdrop
{
    class SomeModel
    {
        public int UserId { get; set; }
        public string Name { get; set; }
        public DateTime BirthDate { get; set; }
    }

    public class SeriLogger
    {
        public static void Main(string[] args)
        {
            string connName = "db-connection-string";

            Log.Logger = new LoggerConfiguration()
                .MinimumLevel.Debug()
                .WriteTo.LiterateConsole()                 
                .WriteTo.RollingFile("AppLogs\\SomeCoolApp-{Date}.txt")
                .WriteTo.MSSqlServer(connName, "some_cool_app_log")
                .CreateLogger();

            try
            {
                Log.Information("Some important log information!");

              var someData = new User { UserId = 256, Name = "Ocean Airdrop", BirthDate = DateTime.Now };

                Log.Information("Some more logging: {@User}", someData);
            }
            catch (Exception ex)
            {
                Log.Error(ex, "Something went kaboomski!");
            }
        }
    }
}

Contact Me:  ocean.airdrop@gmail.com

Sunday, 29 January 2017

Data Modelling and Diagramming Database Schemas

So, I have recently been designing a schema for a new database system. As everyone knows, this is a technical process and requires a good understanding of the problem domain to make sure you collect all the business concepts and relationships.

For these kind of things, you can't beat a good entity relationship diagram to visually see the makeup of your database tables and the relationships between them.

This post is about the "Toad Data Modeller" application which is what I've been using to model my database design. It's such a great tool that I think it should be present in the toolbox of any DBA.

Now, I know Microsoft includes a rudimentary diagramming tool in SQL server. Just right-click on "Database Diagrams" and add the tables you want to include.

But this feature hasn't been shown any love since SQL Server 2000 and the diagrams are very simple. Microsoft Visio is another tool I've previously used and it allows you to create an ERD of your database, although I think you may need the professional version to reverse engineer an existing database. It's not intelligent in any way apart from creating a pretty diagram.

But my new favourite tool to use is Toad's Data Modeler.

Below is an example screenshot of Toads Data Modeler on the sample "AdventrueWorks" database.

It allows you to reverse engineer (read in) an existing database, although the freeware version (the one I'm using) only allows you to reverse engineer a maximum of 25 tables at a time.

Once the model has been loaded you can easily add foreign keys between tables, add unique constraints, and basically ensure there are no floating tables in your design. You can hover over constraints to see which database fields they are attached to. If you have made changes to the actual database schema under the covers, you can "Update the model from the database". And if you have made any changes in the app, you can export the SQL.

In short, it's awesome! It's a great tool to use to scan over your tables to make sure all the required relationships of each individual table is met. Oh, and did I mention it works with all the major database systems. And its free!

If you haven't got it download it from here.


Contact Me:  ocean.airdrop@gmail.com

Sunday, 8 January 2017

Another Year. Another Year's Checklist!

Ahhh yes… Another year! Another fresh start and set of goals for the year. So many good intentions.

On this blog I try to detail my meandering adventures in technology, so I thought I'd write up the tech-subjects I've had in the back of my mind but never got around to looking into.

You never know, maybe this is the year I can tick some of them off!!

Truth be told, I have had this laundry list of things-to-do-and-learn swirling around my head from last year. And maybe the year before as well.


  • C++ 17 will be released this year. I've been keeping up with all the new toys that have been added to the language since C++11. Even though I don't code in C++ anymore (C# is my daily driver) I still have a fondness for C++. It would be good to have a refresher on some of the newer language features.

    Although, it seems like C++20 is going to be the BIG release with modules (finally) and async/await billed as features!

  • Install and play with .NET Core on Linux and port a simple application. .NET Core 1.0 was released last year and I have not yet played with it. Although in my defense they do say to wait until version 2 of a product. You know, for the bigger bugs to be ironed out.

  • Install Linux (again) on one of my machines!! If I am going to test and play with .NET Core I will need to have Linux anyway. Is this the year I finally say goodbye to Windows? I say this every year. And the answer is always no! I love Visual Studio too much!

  • Take a look at the Nim Programming Language. There was a time when I wanted to check out Lua. That urge has now been replaced by Nim.

  • I have had an idea for a side-project for ages. This project would be a "Windows Explorer IMDB Overlay" over the filesystem and would link to IMDB. This TMDbLib API looks like it will do just the job. Its an API for the themoviedb

  • I would like to play around with SkiaSharp and UrhoSharp. They are both graphics libraries. Skia is a C++ open source 2D graphics library and is used in Chrome, Firefox, Android etc. SkiaSharp is a C# port. Urho3D is a cross-platform 2D and 3D game engine. UrhoSharp is a port to C#
  • .
  • Pay some attention to my Amazon EC2 instance. I have a server idling in the sky and it would be good to do something useful with it.

  • Think of an IoT project to do on my Raspberry Pi 2 device. I did have an idea to try and link up my go-pro to it. I did find a website that detailed the go-pro wifi commands and thier general query structure.

  • Play more xbox! (Why isn't this top of the list?)

  • Read up on algorithms.

  • Read the book CODE by Charles Petzold. It's an old book but it's supposed to be a nice walk through of the basic concepts of computers. I've been meaning to read it for ages now.

  • That reminds me! Read more books. I already have an Audible account but I need to set aside more time to physically read. You know. The old-fashioned way. :)

  • Re-Install and play with Gimp and Blender (...Again, hopefully this time I will grok them)

  • I've been meaning to look into Vue.js and Ractive.js. They are both light HTML templating engines and they both use moustache style templating. They both look good and if I ever do a client-side web project I hope to look into these.

  • Play around with PostgreSQL.. I use SQLServer daily and am very adept with it but I have never used PostgreSQL which is just as capable. There is a windows version of the database here. I can't believe the download is only 137MB. In comparison, the download for SQL Server Management Studio (just the interface) is knocking on a gigabyte!

  • On a side note (while were talking of databases), try to think more "set based" than "procedural based" when writing SQL queries. Why do I always think in cursors instead of set based operations? Damn my imperative mind!

  • Finish off my Xamarin side-project and post it on my GitHub. Or at the very least post what I have done so far.

  • Python. If I need to knock up a quick throw-away script (say to do something with the filesystem) I want to try and remember to use Python (just to keep my toe in).

  • Take another look at TypeScript now its reached 2.1! Version 2.1 now compiles async/await code down to EcmaScript 3 without needing Babel! Yay!

  • Take a look at the Kotlin Programming Language! Apparently you can write Android apps in it now!

  • Unity can run on all platforms but can you use it for a line of business app? Probably not, but the scripting language they use is C# and it would be interesting to dig into.

  • .NET Standard 2.0 is due to come out this year (apparently when Visual Studio 2017 drops). I deffo want to play with this. Is it the future of .NET?

So there it is. In short, I seem to have a lot of spinning plates of "things I hope to get around to do/read up on/investigate". There’s a lot out there to learn. I guess that's one of the reasons I have a not-to-do list!


Contact Me:  ocean.airdrop@gmail.com

Sunday, 13 November 2016

.NET Concurrent Collections

Oh ConcurrentQueue, where have you been all my life! - Why have I only just found out that you exist?

I know, I know… It seems I’m late to the party.

But it turns out that the new System.Collections.Concurrent namespace was only introduced in .NET Version 4.0

It’s no secret that when it comes to data processing, adding more threads mean you can process data faster. But adding multiple threads means data races. It means you need to make sure you have a mutex/lock setup around the shared data structure. This ensures that all access to this central data is tightly controlled. Essentially, it means only one thread at any one time should be able to access the data.

Now, I have previously rolled my own thread-safe locking queue which is always a scary concept (due to data-race conditions). I have used it in multiple projects and have many variations.

I have written a Blocking Queue which doesn’t spin on a thread. Essentially all threads will wait on a queue instead of having to Thread.Sleep(). I have also written a thread-safe database writer, specific to the database schema needs, where worker threads will pick up work of a certain type (WorkType), then write it to the database in the order it appears in the queue (important). If another thread wakes up and starts taking work from the master work queue it ignores that WorkType as it is being dealt with by the previous worker thread. This means there needs to be intercommunication between threads. It’s tricky code to get right.

Why do I mention all this?

Because these new thread-safe collection classes could simplify a lot of my current code!

I work on many producer/consumer problems where you can have many multiple producers of work and need many consumers trying to keep up! I have a set of utility classes that I bring along with me to every new project.

But the next time I am working on a project, perhaps it’s time to upgrade to .NET 4.0, then refactor my utility classes to use the classes from the new System.Collections.Concurrent namespace.


Contact Me:  ocean.airdrop@gmail.com

Sunday, 9 October 2016

My Winform "HTML Toast Notification" Library

The other day/week/month (gee, where does the time go!), I wanted to include toast style notifications inside a Winform's application I was working on.

Basically, I wanted some kind of on-screen-display (OSD) that notifies the user when an event happens in the app. I also wanted something non-obtrusive so the user didn't have to interact with it (for example, clicking a button to dismiss it).

The only thing was... I couldn't find anything out there that suited my needs.

So, I wrote my own!

After playing around with some web-based toast notifications, I thought it would be neat to be able to display any HTML in a transparent window. After a bit of research, an article on CodeProject helped me to display a transparent window in C#.

From there, I added my own code to display HTML in the window using the excellent HTML Renderer library found here!

This is what I ended up with:

  • You can display any HTML at any location on the screen.
  • You can display any image inside your project.
  • You can display base64 png images inside your HTML.
  • You can specify a timeout for the toast notification.

You can find the code for the sample project on my github page.

Check it out here: https://github.com/OceanAirdrop/WinformsHTMLToastNotification


Contact Me:  ocean.airdrop@gmail.com

Sunday, 5 June 2016

Measuring CPU performance from the CommandLine using PerfMon

In a previous post, I talked about viewing "coarse grained CPU performance" of an application using Process Explorer. It's a great way of checking "at a glance" how a process is performing CPU wise.

However, if you want to get down with the nitty-gritty details of your server's performance, you need to bring out the big guns and use the built-in perfmon utility.

I don't know why, but I always find perfmon fiddly to use. Maybe it's just me! Anyway, in this post I just wanted to describe the batch-file I use to test a server's health.

As you know, you can drive perfmon from the GUI by running the perfmon.exe command from the start menu. When you do, you will be presented with this screen:


To get started, first use the logman create command to setup the perflog. In the instance below I have named the log "ocean_airdrop_log" and assigned 2 counters to it.
logman create counter ocean_airdrop_log -c "\Processor(_Total)\% Processor Time" "\Memory\Pool Paged Bytes"  -f csv -o D:\PerfMonLogs\oceanairdrop 
The two counters I have selected are:
"\Processor(_Total)\% Processor Time"
"\Memory\Pool Paged Bytes"

I am also outputting the results to a .csv file. A .csv file is better for analyzing the output. This .csv file gets outputted to the directory "D:\PerfMonLogs" and starts each file with the name oceanairdrop.

Once you run this command, you see the log created in the perfmon window under "User Defined" collector sets.


The next command sets the sample interval for performance log. In this case I am logging every 60 seconds.
logman update ocean_airdrop_log -si 60
Next, we start the log by running the start command:
logman start ocean_airdrop_log
If you have the perfmon window open, you will see the status of the log change to running.


At this point you will see the log file created in the output directory we have specified.


At this point you can stop here and continue to run this perf log for as long as you need.

In my batch file I make a call to the timeout command. This allows you to pause a batch file for a specified period of time (it's very useful).
timeout /t 300
For example, this command waits for 5 mins (60 secs *5 = 300)


Finally, to stop the perflog from running issue this command:
logman stop ocean_airdrop_log

Full Batch Perf-Script


With that, here's the full batch script:
-- Delete the perflog if it already exists
logman delete ocean_airdrop_log

-- Create the perflog selecting the counters want to include
logman create counter ocean_airdrop_log -c "\Processor(_Total)\% Processor Time" "\Memory\Pool Paged Bytes"  -f csv -o D:\PerfMonLogs\oceanairdrop 

-- Set the sample interval time. (this is every 60 seconds)
logman update ocean_airdrop_log -si 60

-- Start the perflog
logman start ocean_airdrop_log

-- Wait for 5 mins
timeout /t 300

-- Stop the perflog
logman stop ocean_airdrop_log

-- Delete the perflog
logman delete ocean_airdrop_log

Summary


There you have it. Having the above perf-script handy that you can run ad-hoc is very useful when diagnosing server performance issues.
Contact Me:  ocean.airdrop@gmail.com

Sunday, 29 May 2016

Yo! - I'm a Lambda.

I've just been reading up on the new JavaScript/ECMAScript 6 features and in particular the new fat-arrow syntax for lambdas.

It's funny, but whenever I see this syntax () => { } in C# code, my mind always shouts to me "Yo!! - I'm a Lambda". - Okay, I understand that might sound weird... It is weird and I don't know how to stop it!

Let me explain!

What I mean is, as programmers, when we read code our brain automatically identifies the cryptic string of characters and converts them to a mental model of what the code will do on the fly. For example, when scanning code, if you come across a for or while keyword your brain automatically tells you: "oh... there's a loop coming". If you see an if you automatically know there is a condition and a branch coming.

This happens instantaneously without fore-thought. It's like words on a poster. Your mind reads the words even if you're not interested in what the poster is advertising. In fact, put anything in front of yourself with words and your mind just reads it without asking you! It just happens!

In C#, whenever I see the the following syntax () -> {}, my mind always shouts to me "Yo!! - I'm a Lambda" because when I was learning the arrow syntax I engraved that sentence into my sub-conscious. And now... I can't shake it!!

So there I was, reading the new JavaScript syntax for lambda's and it happened again! This is when I thought it would be interesting to write a blog post on lambda's and compare the different lambda syntax's from various languages. You know, to see if my mind would play this phrase across languages.

Quick n Dirty Overview of Lambda's


Way back in the day, when I first saw a lambda I remember thinking, "what the smeg is that?" - It was weird looking code like this: x => { return x + 1; }. I didn't know what the X was doing on the left hand side. It just looked funny... Turns out, it is a lambda!

This is how you read a lambda expression:
  • Everything to the LEFT of the arrow is the function parameters.
  • Everything to the RIGHT of the arrow is the function body.
Remember good old-fashioned functions like this:
int AddOne(int x)
{
   return x + 1;
}
To write that as a lambda it becomes:
(int x) => { return x + 1; }
Notice above, that you can see the outline of the function still. You can still see the function takes an int as a parameter named x. Okay, its on one-line, and the name has gone but the outline is still there. The additional punctuation/syntax that's been added is the => which is where my "Yo!! - I'm a Lambda" gets triggered.

Turns out though that we can simplify this even more. If the compiler can determine the type that's passed in, you don’t need to put it there. So it becomes:
(x) => { return x + 1; }
Now, as this function only takes 1 parameter it turns out you can also leave out the parenthesis which gives you:
x => { return x + 1; }
Finally, you can simplify it even more and leave out the return statement (and remove the parenthesis). Then it becomes:
x => x + 1
This is the transformation:
// From this traditional Function
int AddOne(int x)
{
   return x + 1;
}

// ...to this Lambda expression!
x => x + 1

That's all good and easy to understand but when I used to see the code below it used to throw me.
var sum = () => 1 + 2;
It was the empty parenthesis that did it. But that's just a lambda expression that takes zero parameters and returns the sum of 1 + 2. But it looks odd.

Want to see what lambdas look like in other languages? Turns out, so did I. Read on!

C# Lambda Syntax

Okay, we already know what the C# code looks like. Below is the AddOne function as a lambda expression.

class Program
{
    public delegate int AddOne(int value);

    static void Main(string[] args)
    {
        AddOne lambdaFunc = (int someVar) => { return someVar + 1; };

        lambdaFunc(2);
    }
}

You can test this out with the online C# compiler here: http://csharppad.com/

JavaScript Lambda Syntax

As I mentioned earlier, it was whilst reading up on JavaScript Lambda functions that prompted me to write this blog post. The JavaScript code looks something like this:

var lambdaFunc = (someVar) => someVar + 1;
lambdaFunc(2)

To test this out, simply press F12 in your browser, navigate to the console tab and enter the above code.

C++ Lambda Syntax

In C++ they don't use the fat-arrow. They use the array index operators [] as the lambda operator. This allows you to capture the surrounding variables by reference or by value and is my "Yo!! - I'm a Lambda" trigger. The code looks like this:

int main()
{
    auto lambdaFunc = [&] (int someVar) { return someVar + 1; };
    lambdaFunc(2); // now call the function
}

You can try this out using the online C++ compiler here: http://cpp.sh

Rust Lambda Syntax

This one's a bit weird. The syntax looks funny as they use the pipe operator | which is something I am not familiar with.

fn main() {
    let lambda_func = |some_var: i32| return some_var + 1;
    lambda_func(2);
}

You can try this out using the online rust compiler here: https://play.rust-lang.org. Incidentally, the rust compiler forced me to change all my variables to snake_case which I thought was interesting. I like the way you can enforce variable names to keep the code consistent.

Python Lambda Syntax


I like the way Python is all in your face by putting the keyword lambda front and centre. Here's the code:
lambdaFunc = lambda someVar: someVar + 1
lambdaFunc(2)

Again, you can try this out here: https://repl.it/languages/python3

Ruby Lambda Syntax

Ruby is quite similar to JavaScript, with a couple of subtle differences. Instead of the fat-arrow they use a thin-arrow! Also the thin-arrow comes before the function parameter list, whereas other languages put the arrow operator in the middle. Here's the code:

lambdaFunc = -> (someVar) { someVar + 1 }
lambdaFunc.call(2)

Here is the online compiler to check it out for yourself: https://repl.it/languages/ruby

Swift Lambda Syntax


Unfortunately, I have not tested this code but from the documentation, the Swift definition of a lambda is as follows: Lambdas are typically enclosed in curly braces { } and are defined by a function type () -> (), where -> separates the arguments and the return type, followed by the in keyword which separates the closure header from its body.
{ (params) -> returnType in
  statements
}

Notice, here you define the return type for the function where other languages put the function body. Swift also has the in keyword which proceeds the function statements. Our example would look like this:

{ (someVar) -> Int in
  someVar + 1;
}

Go Lambda Syntax

Strangely it looks like Go doesn't support the lambda syntax and the language designers have no intention to add the functionality to the language.

See here: https://groups.google.com/forum/#!topic/golang-nuts/Kfm4t3TShTY

Wrapping up

If you throw a dart in the direction of a modern programming language today, chances are it will support lambdas! More and more languages are borrowing concepts from functional programming. Lambdas are a great way to pass around functions and simplify code.


Contact Me:  ocean.airdrop@gmail.com

Saturday, 21 May 2016

Windows Command Line Goodies

With the news that Microsoft is bringing Bash to Windows (what a crazy world we live in!), it looks like I'm going to add "bash" to my "one more thing to learn" list.

In the meantime, I thought it would be good to list the common daily windows cmd.exe tools that I use and find useful. You can't beat the command line for performing fast, repeatable, scriptable actions that you might need to schedule or run ad-hoc.

Let's start the list:

Get the hostname of the computer you are on

If you are like me and can be logged into many remote desktop sessions, then it can be useful to find out the hostname of the computer you are on. To do that, the command is: hostname. This means you will never get caught deleting files from the wrong server again! :)

Another simple command is the whoami command which will tell you the user you are logged on as for this session

Change the title of the command prompt window

Okay, you might laugh, but this is more useful than you think. If you have many cmd prompts open doing different tasks, changing the title of the command window helps you quickly identify the window you need.

List all processes running on the local machine

Ensuring that processes are running is essential bread and butter stuff. The tasklist command will give you what you need.

You can also list all processes running and their loaded DLL’s with the -m switch: tasklist -m

List all processes running on a remote machine

The tasklist command above can also list processes that are running on remote machines. Again, very useful. The command is: tasklist /s 10.10.10.10 /u domain\username. You will, of course, be prompted for the user's password

Kill a process running on the local machine

I use this all the time. There is no faster way of closing chrome.exe and all its tabs than from the command line. The command is: taskkill /F /IM pcocessname.exe.

Kill a process running on a remote machine

To kill a process on a remote machine you just need to supply the /s and /u flags to the taskkill command. For example: taskkill /s 10.10.10.10 /u domain\username /IM "appname.exe"

Display all services running on a machine

If you have services that you know should be running, then running net start shows you all services that have been started on your machine.

Pinging and IP addresses

This is the simplest of all diagnostic commands: ping 10.20.30.40. If you want to keep a constant ping going add the -t flag like so: ping 10.109.200.3 -t. I also like the utility fping by Kwakkelflap. You can download it from here.

It allows you to add a date and time to the ping, as well as log the pings to a file. You can even specify the amount of time to wait in-between each ping.

For example, the following command: fping 10.20.30.40 -D -T -c -t 10000 -L hello.txt will ping 10.20.30.40 logging the date and time to a file named hello.txt. It also waits 10 seconds in-between each ping. I like this utility, but one problem I have found is, it doesn't flush the file. This means you need to stop the execution before it writes everything out.

If you want to get your own IP address run ipconfig /all.

Display all IP addresses connected to a machine

If you want to display all IP addresses that are connected to a machine and what process they are connected to then run netstat -a -n. If you want to display all IP addresses connected on a specific port then pipe it through the find command. E.G: netstat -na | find "1234"

PSTools

This should probably be at the top of the list. The PSTools suite is great and my stand out favorite is the psexec tool. It allows you to run any dos command you issue on a remote computer.

For example, if you wanted to run netstat –n to display all IP addresses connected to a remote machine then you could run the following command: psexec.exe \\10.10.10.10 -u domain\username -p password netstat –n

Elevate

Have you ever tried modifying the hosts file only to be greeted with this dialog?

If so, then Elevate is the command line tool you need. This is just one example, but there can be many cases where you need to run something with elevated rights.

Connect to SQL Server database

If you have an SQL server instance you can run queries straight from the command line by using the osql command. For example: OSQL -S 10.10.10.10\SQLExpress -U username -P password -d dbname -Q "select * from [dbname]..[tablename] where columna = 'blah-de-blah'"

You can even send the result to a text file OSQL -S 10.10.10.10\SQLExpress -U username -P password -d dbname -Q "select * from [dbname]..[tablename] where columna = 'blah-de-blah'" >c:\textfile.txt

IP routing and adding a fixed route to your routing table

To view your routing table simply run route print. This will show you all the routes that are setup on your machine. You can add a fixed route to your routing table by running the "route add" command. This can be useful if you want to route traffic in a certain IP address range to a specific destination server. This has been useful to me in OpenVPN environments when I needed to force specific IP packets to a gateway.

To do this, first delete the route before adding it (just to make sure it doesn't already exist). For example: route delete 172.16.0.0. Then add the permanent route by running "route add" with the -p flag set. The -p flag says "make this permanent". For example route -p add 172.16.0.0 MASK 255.255.0.0 10.20.30.40 will create a route whereby and traffic that is destined for the IP range 172.16.x.x will get routed/pushed to the server 10.20.30.40.

Test if a remote server is listening on a specific port number

If you have a server listening on a remote port and you want to check it is accepting incoming connections, then you can use telnet to connect to the server and port. If it fails you will get the following error message back: "Could not open connection to the host, on port 25: Connect failed". This is very useful.

For example, the following command will try and connect to a server on port 80: telnet 10.20.30.40 80

Remote shutdown a SERVER!!!!

Naturally, use this one with caution! It's always a bit scary waiting for the server to come back up. When I do a remote-boot of a server, I keep a constant ping going so I can see when it comes back up

Don't forget, its obligatory when remote booting any server to mutter the Samuel L Jackson line from Jurassic Park: "hold onto your butts!"

Here's the command: shutdown -t 0 -r -f -m \\10.10.10.10

Find out users Logged onto a machine and logging them off

Under certain situations, you might find that you can't rdp onto a server because there are other users logged on. If this is the case you can issue the following command: quser /server:10.10.10.10. This will return a list of username's and and user id's. Following on from this, if you want to log a user off then run the logoff command, like so: logoff 1 /server:10.10.10.10. The 1 here is the user id returned from the quser command.

Time how long it takes a web page to load

cURL is a command-line tool that can communicate over a network using TCP, HTTP, etc. It includes metrics which means you can use it to time how long it takes for a web page to load.

Use it like so: curl -3 -k "https://oceanairdrop.blogspot.com"

Summing up

These commands are all bread and butter stuff but are useful to know to get diagnostic information about your environment. If you are responsible for a number of servers some of these commands can be a life saver. If you are unable to log onto a remote machine (using mstsc.exe) for whatever reason, don't forget about the command line. Being able to run these commands remotely (psexec.exe) is a godsend.

That's about it.... Remember, when issuing any of these commands, don't forget to "hold onto your butts!"


Contact Me:  ocean.airdrop@gmail.com

Popular Posts

Recent Posts

Unordered List

Text Widget

Pages