Technical Difficulties from on Top of the Mountain
2013-12-16
  Staying inside the lines.
When it comes to dealing with binary data, especially binary packets from public interfaces, you can't take any chances. C was built a long time ago when doing things safe was very expensive, and so they chose speed. For a very long time I built systems where speed was essential as well (in computer graphics an extra instruction can be multiplied by a billion), but I eventually moved into more general computing problems, and at the same time computers got thousands of times faster.

In a modern system, random memory access is now the killer. The CPU has cycles to burn. Some of the lessons I've learned on performance recently taught the exact opposite of what was true twenty years ago.

To that end, a modern style of dealing with containers has to be bounded. You need to know where you are, and what your limits are. There's some issue with what operator++() should do when you reach those limits, but the one answer for sure is that it can't just go stepping past and start stomping on whatever's next. To that end, my current libraries have the following concepts for both buffers and containers:

A Range is a beginning and and end.

These never own the storage, they just indicate where it is. This can be used for both the available space to write to, and for the used space containing data within a buffer or container.

A cursor is a range + an iterator.

This is where I've gone a bit past all the other work in C++ containers, but I think this is important. Modern iterators (at least the fun ones), are bidirectional or random access. That means the beginning is as important to keep track of as the end. And copying a cursor should not narrow you to the space you had left, but should allow the copy to head backwards to the front just as easily as progressing to the end.

This also gives us a great data structure for those algorithms like std::rotate() that operate on three iterators.

There have been a lot of people banging around on this for some time. Andrei Alexandrescu wrote a great paper On Iteration that had lots of stuff to say about his implementation of containers and interators for D.

Labels: ,

 
2013-12-02
  The fractured pieces of C++
A lot of early programming in C++ was simply objects and polymorphism. This was pretty powerful stuff for those of us used to C, and we build some pretty big things with it. But after implementing linked lists several times, we began wondering about that other odd uncle in the toolbox: templates.

What I and many others ended up with after diving in there was encumbered lists. This is generally frowned upon by advanced C++ people who point out that there is a standard library with a number of "high quality" containers in them. But we arrived at encumbered lists, because we were making lists of polymorphic objects. The STL containers are all made for uniform types.

To provide a bridge, the boost people threw the PIMPL hammer at it.

As I understand it, the original use of PIMPL was to "hide" full class definition from users of the class (usually at library boundaries), and also to significantly speed up compile times for large systems. Compiling has never been an instantaneous process (sadly). Over my career, I've had to deal with more than one system that took hours to rebuild, so knocking time off is more than an academic curiousity.

Applied to the problem of polymorphic objects in containers, you end up with a PIMPL front-ing object, with a hidden implementation behind an abstract interface. Throw in a smart pointer as well to manage life-cycle and you suddenly have problem of memory coherency, because for your first object allocated you get:

And heaven help you if you have a string or something else dynamic in your object.

I think there's a better way, but its a shame that no one else has actually built anything that gives you the power of containers, the algorythms, but plays well with polymorphic objects.

Labels: , ,

 
Life in the middle of nowhere, remote programming to try and support it, startups, children, and some tinkering when I get a chance.

ARCHIVES
January 2004 / February 2004 / March 2004 / April 2004 / May 2004 / June 2004 / July 2004 / August 2004 / September 2004 / October 2004 / November 2004 / December 2004 / January 2005 / February 2005 / March 2005 / April 2005 / May 2005 / June 2005 / July 2005 / August 2005 / September 2005 / October 2005 / November 2005 / December 2005 / January 2006 / February 2006 / March 2006 / April 2006 / May 2006 / June 2006 / July 2006 / August 2006 / September 2006 / October 2006 / November 2006 / December 2006 / January 2007 / February 2007 / March 2007 / April 2007 / June 2007 / July 2007 / August 2007 / September 2007 / October 2007 / November 2007 / December 2007 / January 2008 / May 2008 / June 2008 / August 2008 / February 2009 / August 2009 / February 2010 / February 2011 / March 2011 / October 2011 / March 2012 / July 2013 / August 2013 / September 2013 / October 2013 / November 2013 / December 2013 / December 2014 / February 2015 / March 2015 / July 2016 / September 2016 / December 2016 / April 2017 / June 2017 / July 2018 / November 2018 / January 2019 / February 2019 / April 2019 / December 2019 / March 2020 / April 2020 / May 2020 / September 2020 / November 2020 / March 2021 / May 2023 /


Blogroll
Paul Graham's Essays
You may not want to write in Lisp, but his advise on software, life and business is always worth listening to.
How to save the world
Dave Pollard working on changing the world .. one partially baked idea at a time.
SnowDeal
Eric Snowdeal IV - born 15 weeks too soon, now living a normal baby life.
Land and Hold Short
The life of a pilot.

The best of?
Jan '04
The second best villain of all times.

Feb '04
Oops I dropped by satellite.
New Jets create excitement in the air.
The audience is not listening.

Mar '04
Neat chemicals you don't want to mess with.
The Lack of Practise Effect

Apr '04
Scramjets take to the air
Doing dangerous things in the fire.
The Real Way to get a job

May '04
Checking out cool tools (with the kids)
A master geek (Ink Tank flashback)
How to play with your kids

Powered by Blogger