This problem can be easily circumvented by using reflections. The following code snippet demonstrates how the current function’s fully qualified name can be obtained using reflection:

And reflection can also be used to obtain the values of member fields (e.g. public and private, etc.):

This is particularly helpful when logging exceptions, as the member variables’ states would be preserved and it might provide valuable information in determining the causes of the exceptions.

We need to be careful where we use the techniques mentioned above however, as reflection typically comes with pretty hefty performance penalties and thus must be used sparsely where logging is absolutely needed. When reflection is used within the exception catching block, performance is usually not a concern. But we must pay attention to ensure not to cause further exceptions within the exception handling code.

I was doing some ASP.NET work and was using customized objects as data sources to bind to DataGrid. For sorting, I used the technique similar to what described in Sorting GridView Using IComparer. When I was testing my grids, however, I noticed that for most of the columns it worked perfectly, but for a few I somehow was greeted with the null reference error.

At first I thought that this was caused by the null items in those columns. After spending quite sometime digging, I realized that this was due to the case sensitivity issue in reflection. I had used the wrong case in the DataField of  the BoundField. And after I changed the spelling to match that defined in the value object, the problem was solved.

VB developers should pay special attention when using reflections, as in VB.Net the language itself is not case sensitive.

As a side note, even though it is possible to set the binding flag to make the member lookup case insensitive, it is not recommended as such approach tend to make the operation less efficient.

In an article earlier, I discussed how to dynamically load an assembly and execute it via the contract of a predefined interface. The technique mentioned there was useful for writing pluggable modules. However, any non-trivial implementations of a pluggable module probably require the communication between the module and the host program (e.g. progress, status, etc). In the previous discussion, this capability was not present.

To achieve dynamic event handling, we will first modify the PlugInInterface as follows:

    public delegate void ProgressChangedDelegate(object state);

    public interface PlugInInterface

    {

        event ProgressChangedDelegate ProgressChanged;

        void Test();

    }

As shown above, an event is added to the interface (note, in VB.Net the delegate declaration is not required, the parameters can be implicitly supplied to the event ProgressChanged, e.g. Event ProgressChanged(ByVal state As Object)).

Next, I will add the event to the PlugInLib:

    public class PlugIn : PlugInInterface.PlugInInterface

    {

        private Hashtable _ht;

        public event PlugInInterface.ProgressChangedDelegate ProgressChanged;

        public Hashtable Hash

        {

            get {return _ht;}

            set {_ht = value;}

        }

        public PlugIn()

        {

            _ht = new Hashtable();

            _ht["Key"] = "Value";

        }

        public void Test()

        {

            if (ProgressChanged != null)

            {

                ProgressChanged("start");

            }

            Console.WriteLine(_ht["Key"].ToString());

            if (ProgressChanged != null)

            {

                ProgressChanged("done!");

            }

        }

The changes in the main program are a bit more complex:

    class Program

    {

        public void Test()

        {

            string path = Path.Combine(Environment.CurrentDirectory, "..\\..\\..\\PlugInLib\\bin\\debug");

            string fileName = Path.Combine(path, "PlugInLib.dll");

            Assembly asm = Assembly.LoadFrom(fileName);

            Type type = asm.GetType("PlugInLib.PlugIn");

            PlugInInterface.PlugInInterface cls = Activator.CreateInstance(type) as PlugInInterface.PlugInInterface;

            EventInfo eventProgressChanged = type.GetEvent("ProgressChanged");

            Type eventDelegate = eventProgressChanged.EventHandlerType;

            Delegate d = Delegate.CreateDelegate(eventDelegate, this, "OnProgressChanged");

            MethodInfo methodInfo = eventProgressChanged.GetAddMethod();

            Object[] args = new object[] {d};

            methodInfo.Invoke(cls, args);

            cls.Test();

        }

        private void OnProgressChanged(object state)

        {

            Console.WriteLine(state.ToString());

        }

        static void Main(string[] args)

        {

            Program p = new Program();

            p.Test();

        }

    }

To hook up the event from the dynamically loaded assembly, we first use Type.GetEvent to retrieve the EventInfo from PlugInLib.dll. Then, a delegate is created and hooked to the OnProgressChanged method within the current running assembly. Finally, methodInfo.Invoke(cls, args) links the delegates with the correctly the parameters with the event handler.

The example shown here can be downloaded here.

There are already many good guides out there illustrating how to construct a weakly coupled system that dynamically loads in modules as needed. In .Net, this can be accomplished using reflection.

However, some people are confused about Reflection versus Serialization, thinking that a class must be serializable before it can be dynamically loaded. This is not the case however. Reflection is typically used to reflect upon the object’s type, member information and dynamically invoke object’s methods whereas serialization is typically used to transfer objects’ states across wires.

Here I will give a very simple example to show how reflection can be used to create pluggable application architecture.

First, let’s create an interface that defines the common functionalities of our plug-ins. It is always desirable to define a common interface for all the plug-in modules as the main application that loads and executes these modules need not to have any information about the plugged in objects.

public interface PlugInInterface

{

void Test();

}

Next, we create a simple pluggable module that is based on this interface.

public class PlugIn : PlugInInterface.PlugInInterface

{

private Hashtable _ht;

public Hashtable Hash

{

get {return _ht;}

set {_ht = value;}

}

public PlugIn()

{

_ht = new Hashtable();

_ht["Key"] = "Value";

}

public void Test()

{

Console.WriteLine(_ht["Key"].ToString());

}

}

In the above example, the module does nothing but adds an entry to a hashtable. The point I want to make here is that since we will be using reflection mechanism to invoke this class, no serialization is required. In fact an hashtable cannot be simply serialized by the Serializable() attribute as XmlSerializer does not serialize objects that implement IDictionary by default (see this article on MSDN on how to get around this).

Here is the main class that loads the plugin assembly using reflection:

class Program

{

public void Test()

{

string path = Path.Combine(Environment.CurrentDirectory, "..\\..\\..\\PlugInLib\\bin\\debug");

string fileName = Path.Combine(path, "PlugInLib.dll");

Assembly asm = Assembly.LoadFrom(fileName);

Type type = asm.GetType("PlugInLib.PlugIn");

PlugInInterface.PlugInInterface cls = Activator.CreateInstance(type) as PlugInInterface.PlugInInterface;

cls.Test();

}

static void Main(string[] args)

{

Program p = new Program();

p.Test();

}

}

Since the plugin implements PlugInInterface, it is desirable to put the interface in its own class library so that we only need to make a reference to the interface class library not the actual plug-in itself.

The full example code can be downloaded here (in C#, Framework 2.0).

In .Net, an XML file or even a CSV file can easily be mapped to an object using proper attributes. Using the reflected object type, such XML or CSV files can easily be deserialized into the corresponding object.

However, sometimes we do not have the luxury in specifying the format of the data file and beyond the common XML and the less common CSV file format, we rarely use serialization technique to retrieve data from the file. For a concrete example, let’s take a look at the following file that represents a free style inventory list:

;—————- itemName = Pen quantity = 10 ;—————- itemName = Paper color = Red size = A4 quantity = 100 ;—————- itemName = Box size = Large quantity = 20 ;—————- itemName = Ruler length = long quantity = 30 ;—————-

As you can see from the list, each inventory item has a few properties that are a subset of all the available properties. Similar data structures can be found in other real world programs. For example, the INI file structure is similar to the example I gave.

This kind of files is typically processed nonreflectively. We can however, still utilize some features with reflection to make a strongly typed object from the text file.

The first step is to create a value object that holds all the possible keys (sample code in C#):

public class InventoryItem { private string _itemName = null; private string _color = null; . . .

public string ItemName { get { return _itemName; } set { _itemName = value; } } . . .

}

Next we can use a parser to parse the text file line by line and use reflection to populate the class members and thus obtain a strongly-typed object.

public void Parse() {

. . . string line = string.Empty; line = fileStream.ReadLine();

do { if (line == null) break;

if (line.Trim().IndexOf(";—-") == 0) { if (_content == null) { _content = new InventoryItem(); } else { _items.Add(_content); _content = new InventoryItem(); } } else { AddMemberInformation(line); } line = fileStream.ReadLine(); } while (line != null); fileStream.Close(); }

private void AddMemberInformation(string line) { string[] ay = line.Split(‘=’); Type t = _content.GetType(); FieldInfo field = t.GetField(String.Concat("_", ay[0].Trim()), BindingFlags.Instance | BindingFlags.NonPublic);

field.SetValue(_content, ay[1].Trim());

}

The trick used here is in procedure AddMemberInformation(). The private fields are retrieved from the InventoryItem class via Type.GetField. Note that in order to match the private variable names in InventoryItem class, “_” is concatenated with the variable name found in the text file. Also, fields retrieved via reflection are case-sensitive. This might surprise those VB.Net programmers as variables in VB.Net are generally case insensitive.

In .Net, distributed computing can be done in a number of ways.

The most widely used way is to use web services, where a remote client would call a publicly exposed web method on a server where the web service is hosted. When using web services, the code of the service method must be physically on the server where the service is hosted. In this scenario the client talks to the server via a proxy class that represents the web service on the server.

Alternatively, we can use .Net remoting to execute code in a distributed environment. Typically, either TCP or HTTP channel is used to communicate object states between client and server. The main advantage of using remoting is that custom sinks can be utilized to incorporate sophisticated data manipulations before the data is sent and received. The amount of data transferred can also be shrunk to minimal if binary formatters are used and thus resulting in very little channel overhead as compared to SOAP messages that typically used in web services.

Object serialization transfers the state of an object, not the physical object itself however. For serialization to work, the object to be serialized needs to exist on both the client and the server. When the object gets serialized, the state of the object is transferred across the wire. And when the object is deserialized, the object on the server side gets created and then populated with the states from data (the serialized object) transferred from the client.

So, in both scenarios, whether using web services or remoting, the object that needs to be used remotely must reside on both ends (a proxy or the actual object).

Autonomous Remote Code Execution

Can we use just one copy (e.g. the client version of the object) of the object to achieve remote code execution in a distributed environment? As it turns out, we can do this using reflection. For a lack of a better name, I call this approach autonomous remote code execution.

The idea here is to create an object on the client side, and physically ship the object over to the server side and the server will execute whatever code that is send over.

I will demonstrate how this can be done using TCP socket. To make the discussion easier, we will use synchronized socket communication model here, the principals illustrated here apply also to asynchronized sockets.

The Client

The client code first creates a TCP socket and reads in whatever .Net application needs to be executed as binary bytes. Then the content is sent through the TCP socket.

using System;

using System.Collections.Generic;

using System.Text;

using System.Net;

using System.Net.Sockets;

using System.IO;

namespace SocketClient

{

class Program

{

public void Send()

{

TcpClient socketForClient;

try

{

socketForClient = new TcpClient("alpha", 10); //server name

}

catch {

Console.WriteLine("Failed to connect to server");

return;

}

NetworkStream networkStream = socketForClient.GetStream(); //Read in the program to be sent

FileStream fs = new FileStream("PrimeGen.exe", FileMode.Open);

BinaryReader br = new BinaryReader(fs);

byte[] bin = br.ReadBytes(Convert.ToInt32(fs.Length));

fs.Close();

br.Close();

StreamReader streamReader = new StreamReader(networkStream);

BinaryWriter streamWriter = new BinaryWriter(networkStream);

try {

string outputString;

outputString = streamReader.ReadLine();

Console.WriteLine(outputString);

streamWriter.Write(bin); streamWriter.Flush();

}

catch {

Console.WriteLine("Exception reading from Server");

}

networkStream.Close();

}

static void Main(string[] args) {

Program p = new Program();

p.Send();

}

}

}

The Server

On the server side, a socket is created and then is blocked listening for the incoming connections. When the client and the server establish the connection, we read in the data as an array of bytes which represent the actual application transferred from the client side.

We then can get the actual object back using reflection. Finally, we can execute the code by jumping to its entry point. In this particular example, it is a .Net program called PrimeGen.

using System;

using System.Collections.Generic;

using System.Text;

using System.Net;

using System.Net.Sockets;

using System.Reflection;

using System.IO;

namespace SocketServer {

class Program {

public void Receive() {

TcpListener tcpListener = new TcpListener(Dns.GetHostEntry("alpha").AddressList[0], 10);

tcpListener.Start(); Socket socketForClient = tcpListener.AcceptSocket();

if (socketForClient.Connected) {

Console.WriteLine("Client connected");

NetworkStream networkStream = new NetworkStream(socketForClient);

StreamWriter streamWriter = new StreamWriter(networkStream);

BinaryReader streamReader = new BinaryReader(networkStream);

string theString = "Sending"; streamWriter.WriteLine(theString);

Console.WriteLine(theString); streamWriter.Flush();

//Note, here I simplified it a bit. Since I know that the

//code is less than 16K, so I can just read everything in

//at once. If the code size can not be predetermined, or it

//is too large for a single transmittion, you will have to

//slice it up and reassemble the data at the server side.

byte[] bin = streamReader.ReadBytes(16384);

//load the assembly sent from the wire

Assembly a = Assembly.Load(bin); MethodInfo method = a.EntryPoint;

if (method != null) {

object[] obj = new object[1] {new string[] {}};

try {

method.Invoke(null, obj);

} catch (Exception e) {

Console.WriteLine(e.Message);

}

}

streamReader.Close();

networkStream.Close();

streamWriter.Close();

}

}

static void Main(string[] args) {

Program p = new Program(); p.Receive();

}

}

}

Client Server in Action

Here is a screen shot of the client and server in action:

Remote Execution Using Windows

Using this autonomous remote execution method, we can easily run the server anywhere by simply copy the server side executable. Just for fun, here is a screen shot of the server running on a FreeBSD server and the client is running Windows XP.

Remote Execution using FreeBSD

Final Thoughts

The autonomous remote code execution method mentioned here can be used in various situations. For instance, when conducting numerical analysis in a distributed fashion, it would typically require setting up the same code on all the computers that participate in the computation. To ensure that the all the computers are setup with the correct version of the code can be quite tedious. Further, whenever the computation task is changed, all the code needs to be re-deployed again across all the machines.

Using the approach mentioned here, we can in theory just install the server version of the code on all the computational nodes. This needs not to be changed over time regardless of whatever the code needs to be run in a distributive fashion. A single machine can be setup to inject the code that needs to be executed collaboratively into all the computational nodes.

Like everything else, this method has its limitations as well. For instance, the entire server instances need to run at a privileged level, otherwise the reflection portion of the server code might fail. Because the server code runs whatever the client tells it to run, running under privileged accounts can be potentially dangerous. However, for scientific computations, this should not be an issue since all the computational nodes are typically contained behind a firewall.

The size of the code needs to be run remotely should be small enough for this method to be efficient, as the bandwidth could become a bottle neck if the code size is big or the number of computational nodes is large or both.

While I only demonstrated running a very simple test program remotely, it is possible to run programs that contain many DLLs and span multiple assemblies. Due to the limitation of our in-memory representation of the code on the server side however, it is mandatory that all assemblies be combined into a single executable before it can be used with the method mentioned here. This can be achieved by using ILMerge when working with Microsoft .Net implementation or mkbundle when using Mono.

As it turned out, while the .net framework 2.0 solved some of the xml serialization problems that plagued developers using framework 1.0, the same problems manifest themselves in different ways in framework 2.0. And the symptom of this problem can be mysterious and hard to identify.

In my recent consulting position, I was developing some code using Visual Studio 2005 to integrate with some third party web services. But when I added a web reference to the service and wrote a simple one line program to test it, I got this exception: (note, I have replaced the class name, project name and path with {…}, so that I don’t have to reveal any information that might be related to the project.)

Server Error in ‘/{class name} Application.

Unable to generate a temporary class (result=1).
error CS0029: Cannot implicitly convert type ‘string’ to ‘string[]‘

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.InvalidOperationException: Unable to generate a temporary class (result=1).
error CS0029: Cannot implicitly convert type ‘string’ to ‘string[]‘

Source Error:

Line 59:

Line 60: /// <remarks/>

Line 61: public {Method name}() {

Line 62: string urlSetting = System.Configuration.ConfigurationManager.AppSettings["WebReference.{service name}"];

Line 63: if ((urlSetting != null)) {

Source File: c:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\{project path}\d8d15552\e16aa6bd\App_WebReferences.kd_c-r9m.0.cs Line: 61

Stack Trace:

[InvalidOperationException: Unable to generate a temporary class (result=1).

error CS0029: Cannot implicitly convert type 'string' to 'string[]‘

]

System.Xml.Serialization.Compiler.Compile(Assembly parent, String ns, CompilerParameters parameters, Evidence evidence) +1389

System.Xml.Serialization.TempAssembly.GenerateAssembly(XmlMapping[] xmlMappings, Type[] types, String defaultNamespace, Evidence evidence, CompilerParameters parameters, Assembly assembly, Hashtable assemblies) +3416

System.Xml.Serialization.TempAssembly..ctor(XmlMapping[] xmlMappings, Type[] types, String defaultNamespace, String location, Evidence evidence) +171

System.Xml.Serialization.XmlSerializer.FromMappings(XmlMapping[] mappings, Type type) +241

System.Web.Services.Protocols.SoapClientType..ctor(Type type) +546

System.Web.Services.Protocols.SoapHttpClientProtocol..ctor() +243

WebReference.{service name}..ctor() in c:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\{path name}\d8d15552\e16aa6bd\App_WebReferences.kd_c-r9m.0.cs:61

_Default.Test() in C:\Documents and Settings\kwong\My Documents\Visual Studio 2005\WebSites\{path name}\Default.aspx.vb:11

_Default.Page_Load(Object sender, EventArgs e) in C:\Documents and Settings\kwong\My Documents\Visual Studio 2005\WebSites\{project name}\Default.aspx.vb:7

System.Web.UI.Control.OnLoad(EventArgs e) +88

System.Web.UI.Control.LoadRecursive() +74

System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +3035

---

Version Information: Microsoft .NET Framework Version:2.0.50727.42; ASP.NET Version:2.0.50727.42

C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\{project name}\d8d15552\e16aa6bd\ App_WebReferences.kd_c-r9m.0.cs

At first, this looked like a permission issue. Since I had run into similar asp.net related permission issues before, this assumption sounded reasonable. After a quick search of internet, I found these two articles (http://support.microsoft.com/kb/896181, http://support.microsoft.com/kb/908158) on Microsoft’s website, and both articles suggested that I had some permission issue, even though the errors were not quite the same.

But after double checking my permission settings and trying on different machines, I was convinced that this was caused by permissions. So what could it be?

I tried to make sense of the error message, and the line “Cannot implicitly convert type ‘string’ to ‘string[]‘” all of a sudden made me believe that this was due to similar issues as I experienced with Visual Studio 2003. Because by default Visual Studio 2005 does not generate the stub class and put it with the web project, but rather dynamically generates it during compile time (because of this, you see file name like “C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\{project name}\d8d15552\e16aa6bd\ App_WebReferences.kd_c-r9m.0.cs” in the error message.), the only way we might be able to correct this problem is to generate the proxy ourselves and forget about trusting the IDE to generate the class for you.

So I used wsdl.exe to generate the proxy, and sure enough, when I inspected the generated class, I found incorrect use of jagged arrarys ([][]) at places where a regular one dimensional arrays were needed. And after correcting these problems, I was able to consume the web service. By the way, when you use wsdl to generate the web service proxy you only need to include the generated class in your project and do not need to worry about adding the web reference, the proxy class is the reference to the web service.