I mean, the sad part is I actually understand why they likely implemented this instead. At this point in Visual Studio’s lifecycle, it’s easier just to poll to check if the UI is unresponsive than to actually root out the cause of the unresponsiveness. It’s sad, actually. And the fun part is, in preparation for this blog article I didn’t even have to try very hard to get that little balloon to pop up. Took me about 30 seconds (and no I’m not going through the trouble of reporting it to Microsoft because it’s not entirely repeatable and I’m skeptical of what the results would be anyway). Go Visual Studio.

I’ll start this off with a quick question for you to ponder the answer to before I go into the details — What is a “property” on an object other than essentially just a variable defined only within the scope of that object? Rather than answer that question just yet, here’s the problem I was scratching my head over. In SSIS, they have some nifty pre-defined events which you can easily attach event handlers to. Thanks to Jamie Thomson’s blog post entitled “Custom Logging Using Event Handlers” I’ve been happily coding away logging routines using the OnPreExecute and OnPostExecute event handlers (among a few others). But, I soon ran into a problem that wasn’t addressed in the blog post, and that was that every event that fires propagates up the call chain, regardless of whether or not the event is handled^[*]. That is to say, if I have an SSIS package that has (”contains”) a task which loads a file into a database, and I define an event handler for OnPreExecute that simply writes a row to an event log that says “Started: “, then once it gets to the file load task, the database will contain the following:

1. Started: Main package
2. Started: File load
3. Started: Main package

Obviously that last row is a bit silly because the main package didn’t just start, it’s in the middle of executing all of its tasks. That’s because the PreExecute event propagated up from the file load task. Now, one might argue that propagating that particular event doesn’t make a lot of sense because you only ever PreExecute once for each task, and it certainly has nothing to do with when the child tasks do anything. But, for the sake of consistency I can forgive that. If that was the wart, then that’d just be a “huh, thats strange” kind of wart. After all, you certainly do want Error and Warning events to propagate up so you can just define event handlers for those in the parents.

So, at this point I figure there has to be some way to prevent the upward propagation of some events, even if I have to specify it per event handler. The first place I looked was in the properties for the event handler. The only thing that looked mildly promising was the LoggingMode property, but that was limited to “Use Parent Setting”, “Enabled”, or “Disabled” - nothing about “handle and stop propagating”. So, the next place I checked was the “expressions” for the event handler. Expressions in SSIS are basically ways of setting properties that are dynamic (ie, can change at/during runtime), and they’re very useful and generally contain most of the properties anyway, plus an occasional extra tidbit. Well, no extra helpful tidbits in this case, though I was positive that was where it would be.

What does any good programmer do when they’ve exhausted their available local resources? Google it, of course. Unfortunately no matter what search string I used (and my Google-fu is pretty good), I couldn’t find anything that would tell me how to do what I want. Plus, the only other SME on SSIS in the company that I’m aware of was unavailable to ask, and I’m about to outgrow using him as a reference anyway. Then I remembered the aforementioned book, and sure enough one of the first things in the index under events was “bubbling”. And in less than 2 minutes of reading, I had my solution.

Now for the solution and then a quick revisit to the question I kicked this story off with. The solution: there is a “system variable” defined at the error handler scope level entitled Propagate, which if you set it to False will not propagate the event to its parent(s). How the heck is a system variable defined only within the scope of a single object (and obviously only useful within that scope) a better choice than a property? Properties are displayed in a nice little grid that’s easily accessible. System variables are hidden behind several clicks in the interface. I’d wager that 95% or more of SSIS packages never have to muck about with them, and none of the dozen or more packages I’ve worked on have ever needed to do so (until now!), in spite of all the functionality I’ve included. Thankfully, I’m not the only one who had such an issue (I didn’t find that blog post until I started writing this one, and he doesn’t have the solution in his post, but Jamie Thomson posted a response which he found).

And to make matters worse I just realized that this means I’ll have to define that variable for all children within a package otherwise I’ll get a lot of “Started: Main package” messages, one for each sub-task (which is a lot, including checking for input, truncating tables, etc.). According to the blog post and response linked at the end of the last paragraph, there’s no way to tell an event handler to not listen to propagated events either. Sigh.

^[*] - it also doesn’t help that Microsoft’s documentation has a very misleading/incorrect diagram showing event propagation stopping at the first handled event

First I’d have to do some definition/demystifying up front: answering the who/what/why’s and all that, because this is a heavy Microsoft crowd, though not so much that they are fanboys or anything like that (well, some might be but they likely won’t attend) it’s just what they know and that’s what the clients run on. I know at least a few people, including some of the partners, are genuinely interested in open source not only as a business strategy but as a community. So I think the first few big talking points in this section would be:

• What is open source? (I’d like to stick mostly with definitions here and not delve too much into history and/or the fractious issue of what constitues “free” or “open source” software)
• What open source projects they might be aware of, even if they don’t know they are open source (Linux, Mozilla, Java (yes, Java), PHP, Subversion, etc.)
• What open source projects affect them that they are likely unaware of (Apache, Python, MySQL, Mediawiki (wikipedia))
• Why open source? (Why do developers give away code? Why use it?)

And that would be the bulk of the presentation. From there I can come up with a more directed approach talking about how my company can leverage the advantages of open source to help increase productivity/add value/etc. Really I just wanted to hear what people thought about the first part and whether or not I was missing anything or perhaps should take some stuff out. Though if anyone has any feedback for what open source could mean for a small IT consulting firm, then by all means, fire away

>>> foo = "foo"
>>> def func1():
...   foo = "bar"
...   print "Inside func1():", foo
...
>>> def func2():
...   global foo
...   foo = "baz"
...   print "Inside func2():", foo
...
>>> print foo
foo
>>> func1()
Inside func1(): bar
>>> print foo
foo
>>> func2()
Inside func2(): baz
>>> print foo
baz
>>>

In func1(), when I assigned the string bar to foo, Python created a new foo, local to func1() which shadowed the original foo. Which is why before and after the function call, the value of foo (the global one) was “foo”.

Inside func2(), I specifically told Python that I wanted to use the global value of foo, not create a new local one like I did in func1(). This is why the value of foo (the global one) is changed when we do print foo after the function call.

int isnumber(char *s)
{
    size_t i;
    size_t len;
    int period = 0;

    /* Skip over white spaces. */
    while (isspace(*s))
        ++s;

    if (*s == '+' || *s == '-')
        s++;

    i = 0;
    len = strlen(s);

    if (len == 0)
        return 0;

    for (i = 0; i < len; ++i) {
        /* Only one period is allowed. */
        if (!period && s[i] == '.') {
            period++;
        }
        else if (period && s[i] == '.') {
            return 0;
        }
        /* Ignore everything after a space. */
        else if (isspace(s[i])) {
            return 1;
        }
        else if (!isdigit(s[i])) {
            return 0;
        }
    }

    return 1;
}

Let me know what you think and what could be improved. I’ll appreciate any input, whether it’s stylistic, functional, security-related, performance-related, etc.

On the current client I’m designing stuff for, we’re using MS SQL Server 2005’s SSIS toolchain, which is a great idea with mediocre execution. For those who are unfamiliar with SSIS, it’s basically the evolution of their DTS package that came with SQL Server 2000, which is designed to simplify the ETL process of getting data from a source (or various sources) into the destination database after all sorts of cleansing and transformation. The DTS package that shipped with SQL Server 2000 was apparently pretty warty, a trait that is common amongst first-gen Microsoft products, and so SSIS is apparently notably cleaner and nicer. But, as most of us are aware, that doesn’t mean it’s wart-free, and I’ve found a nasty wart or two.

So, for the product I’m working on we have to load lots of files into the database, sometimes batches at a time. They’re segregated into different folders based on what kind of input files they are, and you need to be able to load all of the files in the folder that match a specific file pattern. Great, Microsoft provides a nice “control flow item” named the “Foreach Loop Container” which does exactly that. It looks just like a box with a little label on it where you can place other “control flow items”, and in my case I just use the “Data Flow Task” control item, because that’s what takes data from one place and transforms it and ships it to another. So, since Foreach Loop Containers are necessarily looping over files in a given folder (really it’s not as generic as the name sounds, it’s a “Foreach File in a Folder Container” really), you’d figure that there’d be a sensible way to hook up any file sources within it to, you know, the current file being processed. Well, there’s not. To get to it, you have to define a local variable for the container in the container definitions, and that will be re-set each time it starts a new file. That by itself wouldn’t be so bad if there was a straightforward way to make a data source that simply looked at that variable and was usable that way, but there’s not. In fact, to make it even worse, there is a specific “connection” type called the “Multiple File Connection”, which would work perfectly with this if it were designed to work cooperatively with the Foreach container, but no. The way Microsoft wants you to accomplish it is to set that local variable and then configure a flat file connection pointing to any random file (you have to point it to a file first to configure the output columns), and then tell it to ignore the actual file location you used in setup by overriding the ConnectionString property of the connection object. Of note is that if you should decide that you want to go back and change some of the output column formatting you have to once again go through the whole process of picking one of the files (even though it doesn’t actually use that file, it uses the ConnectionString property which we’ve overriden), then modifying it and then saving it. Luckily you don’t have to re-edit the property again.

All this discussion over what should be a pretty simple task. And another quick, but equally ugly wart is: in SSIS you can’t have a delimited input file that has a variable number of columns that will get parsed nicely for each line (and fill in blank/null outputs for the columns it fails to see). So if I have a CSV file with only 4 columns on the first line and 8 on the next, then my only option in SSIS is to set the number of columns to 4 and then write a script to process the lines that have separators still left in them.

This isn’t the case when we have a call involving a remote leg, where all RTP traffic is running through the Asterisk system. Since we want to have a large RTP pool for local calls and RTP pairs are chosen randomly, we’re unable to simply use a firewall to limit how many connections can be made to the system. Doing so would risk having the external phone instructed to use a firewalled port and therefore lose half of the audio stream.

To address this, I submitted a patch for Asterisk to have a concept of two RTP pools. I left the current pool and its configuration alone and added the ability to optionally specify a “remote” pool. The patch comes in two parts. The first enables configuring the remote RTP pool (specified by the remotertpstart and remotertpend config vars in rtp.conf) as well as providing new functions to create an ast_rtp struct using the remote pool. By default, everything acts as before — the remote pool is the same as the local pool. The second part changes chan_sip to make use of the new remote RTP pool if it detects a remote leg and localnet/externip are set in sip.conf.

Since I think this functionality could be useful outside of my work environment and I’m fairly new to fiddling with Asterisk’s internals, I’d appreciate any feedback on the patch itself and/or actual usage of the feature. As implied above, my patches will only affect SIP calls. Support for other channels would have to be added (if it makes sense).

Most of the initial work was doable within an hour or two, just laying out the models and such. The built in admin interface is really nice, though there was one case where I thought I could use the edit_inline property on some of the ForeignKey fields I had, but it doesn’t look like I can flex it like that. Basically, I was hoping the admin interface would allow me to add in the relevant game results/statistics for this model:

class Game(models.Model):
    week = models.ForeignKey(Week)
    home = models.ForeignKey(TeamGame, related_name='home')
    away = models.ForeignKey(TeamGame, related_name='away')

with the TeamGame class being the individual representation of how a team performed in a given game:

class TeamGame(models.Model):
    team = models.ForeignKey(Team)
    points = models.IntegerField()
    yards = models.IntegerField()
    turnovers = models.IntegerField()

I used a similar data model for the poker league tracking site I wrote in Django about a year ago, only since it was a less-strict “many-to-one” (in that case, PlayerGame to Game) relationship instead of the strict one home, one away “two-to-one” relationship we have here, it makes using edit_inline unworkable, at least the way I understand it and the way I used it on the other site. But, writing a custom view to add the data was relatively simple, and I think I could have even just used the generic CRUD views provided, but I just wrote my own instead.

What makes me wonder if this data model/schema is the “right way” to do it is the fact that extracting some of the necessary data to do the rankings gets kind of cumbersome, probably because of all the normalization (though really normalization is supposed to be the big win here, remember). For example, the next page I wrote after the “add a game’s stats” page was one to view the year-to-date statistics of a given team for each week. And it’s not as clean and pretty as I’d like it to be. In fact, it’s kind of warty-looking, though it’s not too bad in the actual ORM syntax, even though I did resort to using the Q() method, because remember, I want the opponent info as well as the actual team’s info from each game that they’re in, which is what the Game objects are pulled for (and then I assign the current team’s TeamGame for that particular Game in the my_tg variable, and the opponent’s TeamGame in opp_tg):

def display_team(request, team_id):
    team = get_object_or_404(Team, pk=team_id)
    teamgames = TeamGame.objects.filter(team=team)
    games = Game.objects.filter(Q(home__in=teamgames) | Q(away__in=teamgames))
    tmp_games = []
    totals = {'pts_for': 0, 'pts_against': 0,
              'yds_for': 0, 'yds_against': 0,
              'to_margin': 0}
    for game in games:
        if team == game.home.team:
            opp_tg = game.away
            my_tg = game.home
        else:
            opp_tg = game.home
            my_tg = game.away
        if my_tg.points > opp_tg.points:
            result = 'W'
        else:
            result = 'L'
        tmp_games.append({'opponent': opp_tg.team,
                          'result': '%s, %s - %s' % (result, my_tg.points,
                                                      opp_tg.points),
                          'pts_for': my_tg.points,
                          'pts_against': opp_tg.points,
                          'yds_for': my_tg.yards,
                          'yds_against': opp_tg.yards,
                          'to_margin': opp_tg.turnovers - my_tg.turnovers,})
        totals['pts_for'] += my_tg.points
        totals['pts_against'] += opp_tg.points
        totals['yds_for'] += my_tg.yards
        totals['yds_against'] += opp_tg.yards
        totals['to_margin'] += (opp_tg.turnovers - my_tg.turnovers)
    num_games = float(games.count())
    averages = {'pts_for': totals['pts_for']/num_games,
                'pts_against': totals['pts_against']/num_games,
                'yds_for': totals['yds_for']/num_games,
                'yds_against': totals['yds_against']/num_games,}
    return render_to_response('rankings/team.html',
                              {'team': team, 'games': tmp_games,
                               'totals': totals, 'averages': averages})

Granted, a good portion of that is all that dict assignment junk so that I can have a much cleaner looking template file with less “business logic” in it and more pure iteration over results. But it just seems like an unhealthy amount of logic for what is pretty darn simple. But it does make for a pretty simple/clean template:

<h2>{{ team }}</h2>

Games
<table>
    <thead>
        <tr>
            <td>Opponent</td>
            <td>Result</td>
            <td>Yards For</td>
            <td>Yards Against</td>
            <td>TO Margin</td>
        </tr>
    </thead>
    <tfoot>
        <tr>
            <td>TOTALS</td>
            <td>{{ totals.pts_for }} - {{ totals.pts_against }}</td>
            <td>{{ totals.yds_for }}</td>
            <td>{{ totals.yds_against }}</td>
            <td>{{ totals.to_margin }}</td>
        </tr>
        <tr>
            <td>AVERAGES</td>
            <td>{{ averages.pts_for }} - {{ averages.pts_against }}</td>
            <td>{{ averages.yds_for }}</td>
            <td>{{ averages.yds_against }}</td>
            <td></td>
        </tr>
    </tfoot>
    <tbody>
    {% for game in games %}
        <tr>
            <td>{{ game.opponent }}</td>
            <td>{{ game.result }}</td>
            <td>{{ game.yds_for }}</td>
            <td>{{ game.yds_against }}</td>
            <td>{{ game.to_margin }}</td>
        </tr>
        </tr>
    {% endfor %}
    </tbody>
</table>

So I like the result on the template side, but seeing as how I had to do something very similar with my poker site in order to get all the game data for the “display player info” page on that site (that is, aggregate stuff into a temp list or dict and churn a lot of data basically), I’m wondering if there’s a better way to do the view without making the template code crazy complex.