Archive

Archive for the ‘bug fix’ Category

Ethics in open source software

28 August 2010 1 comment

This is a slightly edited version of my posting to the group sage-devel. It raises some issues concerning ethics in open source software. An excellent overview can be found in the paper (Grodzinsky et al. 2003) [1]. My response follows, with any email addresses removed.

Hi Mike,

On Mon, Aug 23, 2010 at 1:07 AM, Mike Witt wrote:
> For whatever it’s worth I’d like to say that I emphatically agree
> that more attention to fixing bugs (presumably at the expense of
> adding features) would make Sage *much* more viable from my point
> of view. My point of view being as:
>
> (1) Not a developer, but simply a user.
>
> (2) Not a mathematician, but someone who is (late in the day 🙂
> slowly making my way through the undergrad math/physics
> sequence.
>
> (3) Someone who has tried unsuccessfully to get classmates and
> instructors interested in Sage as an alternative to certain
> other “M”s.
>
> (4) Someone who, not being particularly brilliant with this
> stuff, probably represents more of what a “typical” user
> would look like if Sage ever attained widespread use.
>
> Having said this, I can’t help but wonder what possible
> motivation there could be, among developers, to do something
> like a bug fix release? It sounds like of boring …

My response can be summarized in one word: accountability. As a first step in understanding accountability as it applies to computer software, let’s look at the notion of accountability in civil engineering. When a bridge collapsed, it is possible to trace accountability back to the company/individual who designed that bridge or to the construction company that built that bridge. The design or construction company/individual can be sued for negligence and so on.

But how does accountability come into play in software? First, a software license usually has an indemnification clause(s). An open source license usually has the explicit statement to the effect that the software concerned is provided as is and without any warranty whatsoever. What other factors can be barriers to accountability within the realm of software? Helen Nissenbaum can provide us with some insights into this question in her two papers:

  • Helen Nissenbaum. Computing and accountability. Communications of the ACM, 37(1):72–80, 1994.
  • Helen Nissenbaum. Accountability in a computerized society. Science and Engineering Ethics, 2(1):25–42, 1996.

In both of these papers, Nissenbaum identifies four barriers to accountability with respect to computer software: (1) the problem of many hands, (2) software bugs, (3) computer as a scapegoat, and (4) ownership without liability.

The problem of many hands can be understood when we consider a mishap with a piece of software. A complex software package is likely to be the result of many people. When the use of that software results in a mishap, on whom do we rest accountability? This issue is more concerned with closed source, proprietary software than it is with open source software. Individual accountability is built into the process of open source software development where one knows who wrote a patch that was accepted and integrated into the mainline source tree. An open source software contributor enjoys a greater degree of autonomy than is enjoyed by a closed source, proprietary counterpart. But it can be difficult for a bug to be fixed once its open source contributor has moved on or is unwilling to fix the bug.

Next comes the issue of software bugs. If you accept that bugs are inevitable, this raises a problem with regard to accountability for such a mentality can encourage sloppiness in how a software package is developed. The open source community treats the eradication of bugs as a group effort, as a group responsibility. Think of the famous phrase attributed to Linus Torvalds: “Given enough eyeballs, all bugs are shallow.” Once a bug is identified/reported, what can we do to get it fixed? How can we encourage people to be active contributors instead of users? Both groups of people are valuable to the success of an open source software project. But a small active contributor base is not likely to bring the number of open bugs down.

Thirdly, the problem of the computer as a scapegoat is a general problem and it is not specific to open or closed source software. I’ll leave this problem as it is and won’t discuss it any more in the context of open source software.

Finally, the issue of ownership without liability is not relevant to open source software. Any open source license must pass the test of the Open Source Definition, which characterizes open source software as something that is not owned by any one individual or company. Open source software according to my reading of the Open Source Definition is more properly a commons in the sense that the open air is a commons for anyone to enjoy, or that a public park is a commons for the use and enjoyment of the people of a community. There is no notion of anyone or any company owning an open source software package in the same way that no one owns the air or the public park. But without ownership, how does accountability comes into play in open source software? I think that comes down to one’s sense of ethics as an open source contributor. How can a person be a responsible open source contributor?

[1] F. S. Grodzinsky, K. Miller, and M. Wolf. Ethical issues in open source software. Journal of Information, Communication and Ethics, 1(4):193–205, 2003.

Advertisements

Sage humour

24 January 2010 Leave a comment

Developing Sage can be serious fun and Sage has been used for serious work in mathematics and computer science. But this doesn’t mean that you can’t use the CAS for some light-heartedness. Observe the following Easter egg:

sage: import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

You could also execute the above command from within a Python or IPython shell. While managing the release of Sage 4.3.2.alpha0, I noticed the following humourous bug report from Tom Boothby at ticket #5501:

The following explodes in sage 3.4:

sage: R = RealIntervalField(4000)
sage: s = 1/R(3)
sage: t = loads(dumps(s))
Traceback (most recent call last):
  File "pikltest.py", line 6, in 
    t = loads(dumps(s))
  File "sage_object.pyx", line 623, in 
sage.structure.sage_object.loads (sage/structure/sage_object.c:6159)
RuntimeError: ('Unable to convert number to real interval.',
 , 
(Real Interval Field with 4000 bits of precision, 
'[a.lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalal0@-1 .. 
a.lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalalala
lalalalalalalalalalalalalalalalalalalalalalalalalalalg@-1]', 32))
invalid data stream
invalid load key, 'x'.
Unable to load pickled data.

Furthermore, it dumps the contents of dumps(s) to the console, which I’m told is a no-no because when one uses ~20kbits of precision with 24 processes via @parallel, the error messages are ridiculously huge.

On a personal note, I’d prefer if my CAS didn’t stick its fingers in its ears and chant “lalalala…” whenever it doesn’t like what I’m doing. This is *not* how a mature system should behave.

How to patch a Sage package

20 January 2010 12 comments

What is an spkg?

The abbreviation “spkg” stands for “Sage package”. The directory SAGE_ROOT/spkg/standard contains spkg’s. In a source install, these are all Sage spkg files (actually .tar or .tar.bz2 files), which are the source code that defines Sage. In a binary install some of these may be small placeholder files to save space.

Sage packages are distributed as .spkg files, which are .tar.bz2 files (or tar files) but have the extension .spkg to discourage confusion. Although Sage packages are packed using tar and/or bzip2, please note that .spkg files contain control information (installation scripts and metadata) that are necessary for building and installing them. For source distributions, when you compile Sage the file SAGE_ROOT/makefile takes care of the unpacking, compilation, and installation of Sage packages for you. For more information on the structure of .spkg files, please refer to the Sage Developer’s Guide in your local installation of Sage at

SAGE_ROOT/sage/doc/output/html/en/developer/index.html

If you cannot locate that file in your local installation of Sage, you might want to consider (re)building the standard Sage documentation using this command:

SAGE_ROOT/sage -docbuild all html

or see the Developers’ guide, especially the section “Producing New Sage Packages”. Additional Sage packages can be found on the Sage website.

How do I patch an spkg?

Make sure you are familiar with the structure and conventions relating to spkg’s; see the section Producing New Sage Packages for details. Patching an spkg involves patching the installation script of the spkg and/or patching the upstream source code contained in the spkg. Say you want to patch the GAP package gap-4.4.10.p17. Note that “p17” denotes the patch level of the spkg. The installation script of that spkg is

gap-4.4.10.p17/spkg-install

In general, a script with the name spkg-install is an installation script for an spkg. To patch the installation script, use a text editor to edit that script. Then in the log file SPKG.txt, provide a high-level description of your changes. Once you are satisfied with your changes in the installation script and the log file SPKG.txt, use Mercurial to check in your changes and make sure to provide a meaningful commit message.

The directory src/ contains the source code provided by the upstream project. For example, the source code of GAP 4.4.10 is contained under

gap-4.4.10.p17/src/

To patch the upstream source code, you should first copy the relevant file under src/ over to the directory patches/. Then edit that copied file rather than the relevant file under src/. Once you are satisfied with your changes to the copied file under patches/, generate a unified diff between the original file under src/ and the corresponding copied (and edited) file under patches/. Save the unified diff to a file with the same name, but use the file extension “.patch”, and place the diff file under patches/. Note that both of the directories patches/ and src/ should not be under revision control. The Mercurial configuration file .hgignore should contain the following two lines

patches/
src/

Make sure that the installation script spkg-install contains code to copy over the patched file under patches/ to the relevant place under src/. For example, the file

gap-4.4.10.p17/patches/configure.out-ia64

is a patched version of the file

gap-4.4.10.p17/src/cnf/configure.out

The installation script gap-4.4.10.p17/spkg-install contains the following conditional to test under which conditions to copy over the patched file to the relevant place under src/:

# If we're on an Itanium Linux box, we overwite configure.out 
# with a slightly modified version.  The modified version has
# all -O2's replaced by -O0. See 
# http://www.gap-system.org/Faq/Hardware-OS/hardware-os8.html
# On the San Diego Super computer `uname -p` is unknown, but
# uname -a includes ia64 in the output.  So this is a better
# detection method. Note that GAP was "fixed" to work fine on
# Itanium without this -O0 hack, but with GCC-4.4.0, GAP 
# mysterious stopped working again.  So we revert to using -O0.
if [ `uname` = "Linux" ]; then
   uname -a |grep ia64
   if [ $? = 0 -o `uname -p` = "ia64" ]; then
       cp patches/configure.out-ia64 src/cnf/configure.out
       echo "The file configure.out was patched for SAGE!" > src/cnf/configure.out.README
   fi
fi

Now provide a high-level explanation of your changes in SPKG.txt. Once you are satisfied with your changes, use Mercurial to check in your changes with a meaningful commit message. Next, increment the patch level of the spkg by one, e.g. rename gap-4.4.10.p17 to gap-4.4.10.p18. If you want your patched spkg to be reviewed and available in the next release of Sage, you need to compress it using tar and bz2. Under Linux and OS X, you can compress your patched spkg as follows:

tar -jcf gap-4.4.10.p18.spkg gap-4.4.10.p18

As part of your request for your patched spkg to be reviewed, you need to provide a URL to your spkg on the relevant trac ticket and/or on an email to the relevant mailing list. Usually, you should not upload your spkg to the relevant trac ticket.