tag:blogger.com,1999:blog-119405922024-03-08T20:37:14.502+00:00The Darke SidePython people refer to Perl as 'the dark side' - geddit?Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.comBlogger81125tag:blogger.com,1999:blog-11940592.post-90411926348992320452012-09-06T08:47:00.001+01:002012-09-06T08:47:37.351+01:00It's a bazaar
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";">I never
really agreed with Eric Raymond's seminal work "The Cathedral and the
Bazaar".<span style="mso-spacerun: yes;"> </span>He was making the point
that closed source software development takes place in a cathedral-like
environment, i.e. hierarchical, whereas open source development was more like a
bazaar where everyone is equal.<span style="mso-spacerun: yes;"> </span>I can
name quite a few open source projects that are hierarchical, complete with cult
figures and religious fervour.<span style="mso-spacerun: yes;"> </span>The
mobile market though, <i style="mso-bidi-font-style: normal;">now</i> I can see
Raymond's point.<o:p></o:p></span></div>
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";">Fear,
Uncertainty, and Doubt (FUD) was a phrase I first heard relating to a certain
purveyor of large blue mainframes.<span style="mso-spacerun: yes;"> </span>It
could equally apply to any large corporate player.<span style="mso-spacerun: yes;"> </span>"No one ever got fired for choosing
IBM" was a familiar phrase in the 1970s, replace IBM with Microsoft and
that brings it up to date.<span style="mso-spacerun: yes;"> </span><o:p></o:p></span></div>
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";">But not in
the mobile market.<span style="mso-spacerun: yes;"> </span>Whereas corporates
buy their IT systems based on measured evaluations and empirical evidence (and
if you believe that you should not be in sales).<span style="mso-spacerun: yes;"> </span>Mobile devices are as much bling as
work-a-day tools.<span style="mso-spacerun: yes;"> </span>The market has many
more players, and lacks the religious loyalty familiar elsewhere.<span style="mso-spacerun: yes;"> </span>Windows dominates the desktop; will Windows 8
penetrate the mobile market?<span style="mso-spacerun: yes;"> </span>Microsoft is
now selling in the bazaar.<o:p></o:p></span></div>
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";">Nokia's
recent launch of their Lumia 920 has had reasonable reviews, it is early days
and I'm not sure why Nokia's shares plummeted by 11% after the launch – what
did the City expect?<span style="mso-spacerun: yes;"> </span>Microsoft's shares
were virtually unchanged.<span style="mso-spacerun: yes;"> </span>Despite Nokia
being closely familiar with Microsoft, several other manufacturers will offer
Windows 8 telephones, including those also offering Android.<span style="mso-spacerun: yes;"> </span>Microsoft cannot afford to turn away players
like Samsung, but where does that leave Nokia?<span style="mso-spacerun: yes;">
</span>Their new telephone has some interesting technical innovations, but I
would love to see it running Android.<span style="mso-spacerun: yes;"> </span>It
won't take long for the competition to catch-up, but I don't find the idea of
an electronic wallet appealing, with opportunities for NFC pick-pockets.<o:p></o:p></span></div>
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";">Windows has
yet to get that critical mass required to make a Windows phone cool.<span style="mso-spacerun: yes;"> </span>Meanwhile it has to shout against all the
other traders in the bazaar.<o:p></o:p></span></div>
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";">All eyes now
on Apple – with the attributes of the cathedral – and the iPhone 5.<o:p></o:p></span></div>
Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0tag:blogger.com,1999:blog-11940592.post-24768835947757927882012-09-05T14:10:00.002+01:002012-09-05T14:10:59.362+01:00The march of mobile
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";">Those of you
who reside on another planet might have missed the largest change to the IT
industry since the release of the IBM PC – mobile computing.<span style="mso-spacerun: yes;"> </span><o:p></o:p></span></div>
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";">There are
parallels with the mid-1980s.<span style="mso-spacerun: yes;"> </span>Early
versions of PC-DOS were dire, and so were the first few versions of
Android.<span style="mso-spacerun: yes;"> </span>There are lots of differences
though; there is a solid, mature operating system at the heart of Android –
Linux, whereas an even older BSD UNIX is the stable base of iOS and OSX, the
Apple operating systems.<span style="mso-spacerun: yes;"> </span><o:p></o:p></span></div>
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";">The fight
between UNIX-Linux-Windows is an old one and is continuing in the mobile market
today, although the products are very different from the originals.<o:p></o:p></span></div>
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";">Comparing the
approach of each supplier is interesting.<span style="mso-spacerun: yes;">
</span>Apple does not license its software to anyone, and is the sole supplier
of hardware.<span style="mso-spacerun: yes;"> </span>This means they have
complete control over everything from manufacture through marketing, sales and
support, and even the programming language used for applications (ObjectiveC).<span style="mso-spacerun: yes;"> </span>Software suppliers only have to test on a
small range of hardware, and are closely controlled.<o:p></o:p></span></div>
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";">Google, who
produce Android, have taken the Open Source route.<span style="mso-spacerun: yes;"> </span>Linux, and other Open Source software, has
long been a favourite of device manufacturers.<span style="mso-spacerun: yes;">
</span>Why?<span style="mso-spacerun: yes;"> </span>Because they are free to modify
it to run effectively on their own devices.<span style="mso-spacerun: yes;">
</span>Software components like telephony are usually proprietary and can be
plugged into a modified Linux.<span style="mso-spacerun: yes;"> </span>Most of
Android is Open Source, and this enables even start-ups to be involved and
encourages innovation.<span style="mso-spacerun: yes;"> </span>The range of
Android devices is huge, some software testers have a stash of over 400 devices
on which to test their code, and that will have to expand with TVs and games
consoles using Android more and more.<o:p></o:p></span></div>
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";">Android
application development mostly uses Java.<span style="mso-spacerun: yes;">
</span>Google have avoided issues with Oracle (who now own Java) by using their
own runtime environment.<span style="mso-spacerun: yes;"> </span>Anyone can
produce applications for Android, on any subject, including some distinctly
dodgy ones.<o:p></o:p></span></div>
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";">Unusually,
Microsoft occupies a middle ground.<span style="mso-spacerun: yes;"> </span>They
are closely allied with Nokia, which Google described as "two turkeys
don't make an eagle", yet many other manufacturers have licenses to
produce Windows phones, including Samsung, the largest.<span style="mso-spacerun: yes;"> </span>Whether the Windows 8 interface will appeal
or not remains to be seen, but no one should underestimate Microsoft.<span style="mso-spacerun: yes;"> </span>They only have about 3% of the mobile market
right now, but this is an incredibly fast moving world.<span style="mso-spacerun: yes;"> </span>Microsoft have several advantages: their user
interface is familiar to most people for a start.<span style="mso-spacerun: yes;"> </span>There have been mixed reviews of the UI for
Windows 8, but can anyone remember a Windows Beta release that has been any
different?<span style="mso-spacerun: yes;"> </span>Not everything that Microsoft
touches turns to gold, remember Vista, and their Windows CE attempts "also
ran".<o:p></o:p></span></div>
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";">Microsoft
Office dominates corporate office applications, yet none of the
"compatibles" on Android are close.<span style="mso-spacerun: yes;">
</span>That might change, there is certainly an opportunity for Adobe there,
but in the short term Microsoft Office could be the killer app.<span style="mso-spacerun: yes;"> </span>Personally I have doubts whether that
monolith will perform well on mobile devices, but we shall see.<span style="mso-spacerun: yes;"> </span>Hardware is getting more powerful day-by-day,
with quad-cores on mobiles being the norm for high-end devices.<span style="mso-spacerun: yes;"> </span><o:p></o:p></span></div>
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";">The position
of .Net as a development environment is interesting, in that it will run on all
three operating systems using a layer called Mono.<span style="mso-spacerun: yes;"> </span>This makes cross-platform development a
possibility, and its performance compares favourably with Java, even on Android.<o:p></o:p></span></div>
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";">I have been
working with Ian Wallington on producing an Android course "Developing
Android Applications" (QAANDDEV), and we have been struck by how fast
changing the environment is.<span style="mso-spacerun: yes;"> </span>We have
been in IT a long time and have worked in many environments between us, but the
Android world takes some beating for the rate of change.<span style="mso-spacerun: yes;"> </span>There is at least one new version of Android
each year – that is not so different to iOS, but that is not the only
variable.<span style="mso-spacerun: yes;"> </span>Because development
environments and tools come from different suppliers they are not co-ordinated
with the Android release - changes and new products come at an astonishing
rate.<span style="mso-spacerun: yes;"> </span>Devices we bought at the start of
this project six months ago are now uncool and out-of-date.<span style="mso-spacerun: yes;"> </span>During the course development I had to scrap
a chapter and start again because the development environment changed.<span style="mso-spacerun: yes;"> </span><span style="mso-spacerun: yes;"> </span>Ian
has been trying to decipher the latest development techniques when there is
little (accurate) documentation.<span style="mso-spacerun: yes;"> </span>I have
found books from usually reliable publishers to be out-of-date, even those
published a few months ago, and full of bugs.<o:p></o:p></span></div>
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";">This dynamic
environment shows no sign of slowing, the odd patent law-suite will just speed
innovation to get around it.<span style="mso-spacerun: yes;"> </span>We had
better get used to it.<span style="mso-spacerun: yes;"> </span>In my view (and
remember that I'm biased) 20<sup><span style="font-size: x-small;">th</span></sup> century monolithic cultures like
Apple and Microsoft will not be able to keep-up with this rate of change.<span style="mso-spacerun: yes;"> </span>Google embraces the culture and is a true 21<sup><span style="font-size: x-small;">st</span></sup>
century company, combining corporate muscle with community effort.<span style="mso-spacerun: yes;"> </span>Our aim is to keep at the forefront of this
pace so we can help our clients exploit the benefits, regardless of which
platform they decide on.<o:p></o:p></span></div>
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 10pt;">
<span style="font-family: "Arial","sans-serif";"><o:p> </o:p></span></div>
Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0tag:blogger.com,1999:blog-11940592.post-82590371282494252232012-07-18T08:23:00.001+01:002012-07-18T08:23:46.709+01:00<h2>
Korn Shell programming and being "dotted"
</h2>
<br />
While teaching a Korn shell course I was asked "where do you stand on exit verses return"?
To be honest I had not realised you could use return outside a function in Korn shell.
<br />
Well you can, but not in Bash, and the POSIX standard says that the effect is undefined.
<br />
<br />
Why would you want to? The effect is exactly the same as exit, unless you have been executed
through the dot command, "sourced", or "dotted". The argument goes that if you use exit then when your
script is invoked using 'dot' then it would exit the caller, whereas return does not.
<br />
I find the argument fundamentally flawed. It is folly to assume that it is safe to execute
any old script in this way, because it breaks a fundamental rule of programming - encapsulation.
<br />
"Dotted" files do not run in their own namespace or environment. Any change, for example a
cd command, will alter the caller. Therefore writing a script to be safe would also involve
restoring any changes.
<br />
Now, what about variables and functions? ANY declared in the called script will overwrite those
in the calling program. If you are unaware of the script you are using ("I don't know if it has
exit in it") then you will be unaware of its variable names and functions. It could do
absolutly anything to your environment (I use the term loosly). An important principle
of encapsulation is that you have your own namespace, of course with 'dot' you do not.
<br />
<br />
It works both ways. Take the following senario:<br />
<br />
<pre>typeset -i x
. myscript
</pre>
<br />
If myscript has a varible called 'x', and it is used for non-numeric text, then the typeset
will have the effect of altering that text to "0". What if x contains a filename in the script?
This could completely alter or invalidate the action of myscript.
<br />
The dot command is designed specifically so that the called code WILL change your current process,
but the code has to be written specifically for that job. Having blind faith that a script will not
trash you current session, and will even work in your environment, is taking a big risk.
<br />
I suggest, in ksh93, that you put this at the head of your scripts:<br />
<br />
<pre>if [[ $0 != ${.sh.file} ]]
then
echo "Do not . this file!" >&2
return 1
fi
</pre>Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0tag:blogger.com,1999:blog-11940592.post-86056389744106031472012-04-29T07:49:00.000+01:002012-04-29T07:53:28.718+01:00Are we loosing C skills? Should we care?A sign of the times. In the last three weeks I have taught two Python courses and one Perl. Last week a C programming course was running with just one delegate. I have not taught C for several years despite being the only trainer for C in the company (I do not include ObjectiveC). This week I will be teaching UNIX programming using C for the first time for four years.
<br><br>
So what? Software moves on, no one uses C anymore. Really?
<br><br>
Perl, PHP, Python, Ruby, are all written in C. Most of Windows and Linux is written in C. Sure, there are versions like Jython, but the use of that variant is relatively small. Programmers of my generation are gradually "logging-out", what happens then? The number of new C programmers seems very low. That will be fine for while, but we are loosing the skills to operate at the lower levels.
<br><br>
Will the systems of the future be able to build on the C base we have, or will everyone be using high level tools? Does it matter? I think it does, the foundations of software should be solid, not top-heavy. I don't think you can really understand UNIX or Linux unless you understand C. Who will pickup the baton?
<br>Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com1tag:blogger.com,1999:blog-11940592.post-55530637609050809492012-03-02T14:06:00.000+00:002012-03-02T14:06:44.643+00:00C-shell: basic features compared to ksh<div><strong>Sign-on files</strong><br /><br />The following files, if they exist, are read at sign-on:<br /><br />/etc/csh.login instead of /etc/profile - interactive (login) sessions only<br />~/.cshrc instead of $ENV<br />/etc/csh.cshrc if supported, not all c shells use this<br />~/.login instead of .profile - interactive (login) sessions only<br /><br />On sign-off the optional file ~/.logout is executed (no equivalent in ksh).<br />In-line scripts<br /><br />To execute a script in-line, that is without creating a sub-shell, use the source command (instead of the . command), as in bash. For example:<br /><br /><pre>source ~/.login</pre><br /><br /><strong>Filename Completion</strong><br /><br />An interactive C shell can complete a partially typed filename or user name. By default this feature is switched off, to turn it on:<br /><br /><pre>set filec</pre><br /><br />When the user types characters followed by <esc>, the shell attempts to fill in the remaining characters of a matching filename, assuming the characters typed are enough to identify a single file.<br /><br />If the filename prefix is followed by EOF (usually <ctrl>d), the shell lists all filenames that<br />match. It then prompts once again, supplying the incomplete command line typed so far.<br /><br />Ending the name with a tilde (~) indicates that a username is required instead.<br />Aliases<br /><br />The syntax for alias in the C shell is slightly different to other shells:<br /><br />alias new-alias existing-command<br /><br />i.e. there is no assignment character (=).<br /><br /><strong>Functions</strong><br /><br />No. There are none.<br /><br /><strong>History mechanism</strong><br /><br />The history list (and the history command) exists as in other shells, however the syntax for retrieving lines from it is different to ksh. The following table refers to lines in the history list:<br /></div>
<br /><div><pre><br />!n The nth line in the history list<br />!-n The nth previous line<br />!! The last line<br />!prefix The most recent line with the specified prefix<br />^xx^zz The last line, with the string xx replaced by zz<br />!n:s/xx/zz/ The nth line, with the string xx replaced by zz<br />!* All the arguments of the last command<br />!$ The last argument of the last command (instead of $_)<br />!^ The first argument of the last command<br />!:n The nth argument of the last command<br /></pre></div>
<br /><div><br />This is also supported by bash.<br /><br /><strong>Variables</strong><br /><br />Pre-defined variables are different from other shells, the most noticeable feature being that they are in lower and upper case. The list of variables is large, here are a few:<br /><br />The exit status of the previous command is $status (instead of $?).<br /><br />The command line prompt is $prompt (instead of $PS1). The contents of which may contain a large range of format characters. It is too huge to list here, see the man pages for details. Some versions may also support $prompt2 and $prompt3.<br /><br />Command line arguments are set in the array $argv (see below).<br /><br />The current working directory is in $cwd (not $PWD).<br /><br />Lowercase versions of path, home, and term also exist. The lowercase version of path is set from PATH, but is only used by the shell. This does not mean that path can be altered without affecting PATH, since the two are kept synchronised. A similar scheme exists with term, home, and others. Only the upper-case versions are exported. Note that path is a list, i.e. the directory names are separated by white-space, not colons.<br /><br /><strong>Setting values</strong><br /><br />Local variables may be assigned values using set (which is more like BASIC than C), for example:<br /><br /><pre>set var = "She sold sea-shells"</pre><br /><br />Note that spaces are optional around the assignment symbol (=) – hooray! The = is mandatory.<br /><br />To create a variable in the environment block (export), use setenv:<br /><br />setenv var "She sold sea-shells"<br /><br />They may be removed using unsetenv.<br /><br />There must be no assignment symbol (=) with setenv.<br />If you have an existing local variable of the same name then that is not overwritten, a new one is created. For example:<br /></div>
<br /><div><pre><br />$ set var=red<br />$ setenv var blue<br />$ echo $var<br />red<br />$ csh # Create a child<br />$ echo $var<br />blue<br />$ exit<br /></pre></div>
<br /><div><br />In the example above, if we had just done<br /><pre> setenv var </pre><br />then the child process would see an empty variable.<br /><br /><strong>Arrays</strong><br /><br />Like ksh and bash, csh arrays cannot be exported. However unlike ksh and bash csh does not allow you to try. Attempting to setenv an array will give an error.<br /><br />An array is initialised from a list (as in bash and ksh93), where the elements are space separated and delimited by parentheses, for example:<br /><br /><pre>set arry = (HPUX AIX DGUX Dynix Tru64 DRSNX SunOS Linux) </pre><br /><br />The old ksh88 method of initialising an array with set –A is not supported.<br /><br />Access an element of the array using the index, counting from 1 (which is weird, since in C we count from zero).<br /><pre><br />echo $arry[3]<br />DGUX<br /></pre><br />We can also access a range of elements:<br /><pre><br />echo $arry[2-4]<br />AIX DGUX Dynix<br /></pre><br />and the whole array using an index of * :<br /><pre><br />echo $arry[*]<br />HPUX AIX DGUX Dynix Tru64 DRSNX SunOS Linux<br /></pre><br />This is the same as printing the array variable (echo $arry).<br /><br />The number of elements in the array is held in a variable with the same name as the array, prefixed with a #, for example:<br /><pre><br />echo $#arry<br />8<br /></pre><br />so getting the last element is simple:<br /><pre><br />echo $arry[$#arry]<br />Linux<br /></pre><br />With other shells we would have to use braces ({…}), which are not needed in this context, but may still be used to delimit a variable name:<br /><pre><br />set money = "dollar"<br />echo "Give me your ${money}s"<br /></pre><br /><br /><strong>Command-line arguments</strong><br /><br />The list of command line arguments is held in the array argv. To get the complete list of arguments, use $argv[*] (instead of $* or $@). The variables $0..$n are still available, but may alternatively be obtained using $argv[n]. A C programmer would also expect a variable called argc, but the number of arguments is held in #argv instead. See also Arrays.<br /><br />The command shift is supported as in ksh and bash.<br /><br /><strong>Modifiers</strong><br /><br />The C shell also supports modifier codes. These are used by suffixing a variable name with a colon (:), followed by the code letter:<br /><br />h remove a trailing path name (similar to dirname)<br />t remove all leading path names (similar to basename)<br />r remove a file "extension" (suffixed preceded by a dot).<br />e remove the filename preceding a file "extension"<br /><br />When used on a whole list or array, they should be preceded by a g (gh, gt, gr, ge) otherwise they will only be applied to the first element. The exception is the modifier code q, which places the extracted data in quotes (thus avoiding $*/$@ differences).<br /><br />For example:<br /><pre><br />set file = /home/user1/Seas/North.c<br /><br />echo $file:h<br />/home/user1/Seas<br /><br />echo $file:t<br />North.c<br /><br />echo $file:r<br />/home/user1/Seas/North<br /><br />echo $file:e<br />c </div>
<br /><div></pre></div>
<br /><div><br />Other variable operations:<br /><br /><strong>Shell Options</strong><br />There are no shell options like ksh and bash have. Instead everything is controlled by shell variables, and many have the same names as options in ksh, like noclobber, ignoreeof, noglob, etc. See the man pages for a complete list.<br /><br />To set a shell option you normally only have to create the variable, you often do not need to give it a value, for example:<br /><br />set ignoreeof<br /><br /><br /><strong>Variable types </strong><br /><br />No, not in csh. (ksh typeset) For example, to convert to uppercase we have to use:<br /><br />set var = `echo $var | sed \ y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/`<br /><br />Reading from stdin<br /><br />To read a variable from the keyboard (stdin), use the < var =" $<">to continue…"<br />$< <strong>Command substitution</strong><br /><br />Command substitution uses back-ticks ($(…) is not supported), for example:<br /><br />set DIR = `ls`<br /><br />Note that this will return a list, and make DIR into an array. This reduces the need for slicing, for example:<br /><br />set now_date = `date '+%a %H %M %S'`<br /><br />returns a four element array, where:<br /><br />$now_date[1] is the weekday<br />$now_date[2] is the hour<br />$now_date[3] is the minute<br />$now_date[4] is the second<br /><br /><strong>Arithmetic</strong><br /><br />Arithmetic commands are prefixed with the @ symbol, which is actually a built-in command and so must be followed by a space.<br /><br />The usual C/C++/Java arithmetic operators, including post-fix increment and decrement (but not pre-fix), are supported, for example:<br /><br />set x # variable must already exist<br />set j # variable must already exist<br />@ x++ # Increment $x<br />@ j = ($j + 42) * $x # Brackets are supported<br /><br /># To continue over > 1 line, escape end-of-line<br /><br />@ now_secs = ($now_date[2] * $hour_secs) + \<br />($now_date[3] * 60) + $now_date[4]<br /><br /><br /><strong>Conditionals</strong><br /><br />As well as the usual conditional statements (see below) the C shell supports a short cut for testing if a variable exists, the $? prefix. Confusingly it returns 0 if the variable does not exist, and 1 if it does. For example:<br /><br />echo $?non_existent_variable<br />0<br /><br />echo $?existing_variable<br />1<br /><br /><strong>Relational operators</strong><br /><br />!= not equal<br />== equal (note : two = characters)<br />> greater than<br />>= greater than or equal to<br />< pattern1 ="~" file ="~" dirname =" $SYS_LPT/$usernm" userdir =" true" dirname =" '$SYS_JOB_'$type" result =" `eval" dirname =" '$SYS_'$type">Loops</strong><br /><br />As in most shells, there are two basic loops, the foreach loop, for list processing, and the while loop. The syntax is familiar but note, like Perl, the foreach loop does not contain an in:<br /><br />foreach variable-name ( list )<br /><br /># loop body<br /><br />end<br /><br /><br />while boolean-expression<br /><br /># loop body<br /><br />end<br /><br />The commands break and continue have the same meaning as in most languages, break exits the loop prematurely, and continue executes the next iteration at once.<br /><br /><strong><br />Unconditional flow</strong><br /><br />The command onintr is similar to trap in ksh – it executes specific code on a signal, but it cannot pick specific signals to trap. There are three forms;<br /><br />onintr use default signal handling<br />onintr - ignore signals<br />onintr label jump to the specified label and continue execution from there. The syntax of a label is: label:<br /><br />Apparently there is also a goto command.<br /><br /><strong>Redirection</strong><br /><br />Standard channel (stdin / stdout / stderr) redirection is similar, but not the same, as other shells. Channel numbers are not used,<br /><br />command > filename redirect stdout<br />command >> filename append stdout<br />command <>& filename redirect stdout and stderr to the same file<br />command >>& filename append stdout and stderr to the same file<br /><br />The only way to direct stdout and stderr separately is by invoking a subshell, for example:<br /><br />(command > out_file) >& err_file<br /><br />Pipes are supported as normal, however co-processes are not.<br /><br />The set command sets shell variables, so to set option noclobber:<br /><br />set noclobber<br /><br />That is, no –o. To override noclobber, append a !, as in:<br /><br />command >! filename<br /><br /><strong>Background jobs</strong><br /><br />Very similar to ksh – in fact Korn "stole" the idea from csh. The jobs and bg commands are supported.<br /><br /><strong>Quote from man csh</strong><br /><br />"<em>Although robust enough for general use, adventures into the esoteric periphery of the C shell may reveal unexpected quirks</em>". You have been warned!<br /><br /><br /></div>Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0tag:blogger.com,1999:blog-11940592.post-50587058157961990512012-03-02T13:33:00.002+00:002012-03-02T13:38:07.990+00:00That Python next/send demo<pre><span style="font-family:courier new;">import glob<br />import os<br /><br />def get_dir(path):<br /> <br /> while True:<br /> <br /> pattern = path + '/*'<br /> count = 1<br /> for file in glob.iglob(pattern):<br /> if os.path.isdir(file):<br /> print count<br /> count += 1<br /> path = yield file<br /> if path: break<br /> <br /> if not path: break <br /> <br /><br />gen = get_dir('C:/QA/Python')<br /><br />print "about to next"<br />print next(gen)<br />print next(gen)<br />print gen.send('C:/QA')<br />print next(gen)<br /></span></pre><br />Gives:<br /><pre><br /><span style="font-family:courier new;"><br />about to next<br />1<br />C:/QA/Python\AdvancedPython<br />2<br />C:/QA/Python\Appendicies<br />1<br />C:/QA\Android<br />2<br />C:/QA\blanket<br /></span></pre>Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0tag:blogger.com,1999:blog-11940592.post-78818659168731357052012-03-02T12:10:00.015+00:002012-03-02T13:33:15.134+00:00A little Python introspectionIn Python we can load a module and assign an alias:<br /><br /><span style="font-family:courier new;">import <em>module-name</em> as <em>alias-name</em></span><br /><br />I was asked how a programmer can find the mapping between an alias and the real module. That was rather beyond the scope of the course, but here goes.<br /><br />The <span style="font-family:courier new;">locals()</span> built-in, and it's sister, <span style="font-family:courier new;">globals()</span>, were mentioned in the course almost as a 'by the way'. In fact both are very useful for introspection. In this case <span style="font-family:courier new;">locals()</span> will give us the name of the module aliases. It also gives us all the other names that are local, but we are only interested in modules. We can use <span style="font-family:courier new;">inspect.ismodule()</span> to get only those names that refer to modules. Watch it: <span style="font-family:courier new;">locals()</span> returns a dictionary where the keys are the names, but the values are the objects themselves - handle with care. Fortunately, stringifying the object gives us a text string like this:<br /><br /><span style="font-family:courier new;"><module 'GetProcs' from 'C:\Python27\lib\site-packages\GetProcs.pyd'></span><br /><br />and it is simple regular expression to extract the names.<br /><br />We still have issues. Most obvious is that any modules used for the introspection (<span style="font-family:courier new;">inspect</span> and <span style="font-family:courier new;">re</span>) are included in the data. In Python 3 we also get issues trying to read the locals dictionary because it changes during a loop (<span style="font-family:courier new;">items()</span> returns an iterator in Py3). We can solve both by putting the code into a different module (although there are other solutions). But then, how can we look at our callers namespace? Simple: <span style="font-family:courier new;">sys._getframe(1)</span>, which exposes just about everything, warts and all. The locals dictionary is avalable through f_locals.<br /><br />So, here is my module, which I named Spection:<br /><span style="font-family:courier new;"><pre><br />import inspect,sys,re<br /><br />def look(): <br /> for name,val in sys._getframe(1).f_locals.items():<br /> if inspect.ismodule(val):<br /> fullnm = str(val)<br /> if not '(built-in)' in fullnm and \<br /> not __name__ in fullnm:<br /> m = re.search(r"'(.+)'.*'(.+)'",fullnm)<br /> module,path = m.groups()<br /> print "%-12s maps to %s" % (name,path)<br /></pre></span><br />The user's code looks like this:<br /><span style="font-family:courier new;"><pre><br />import Spection<br />import glob as wildcard<br />import ConfigParser as parser<br />import GetProcs as ps<br /><br />Spection.look()<br /></pre></span><br />And the output looks like:<br /><span style="font-family:courier new;"><pre><br />ps maps to C:\Python27\lib\site-packages\GetProcs.pyd<br />parser maps to C:\Python27\Lib\ConfigParser.pyc<br />wildcard maps to C:\Python27\Lib\glob.pyc<br /></pre></span>Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0tag:blogger.com,1999:blog-11940592.post-83073618181168383342011-08-17T09:54:00.001+01:002011-08-17T09:55:36.972+01:00Perl - the way aheadWow! I've just been knocked off my feet.
<br />
<br />I'm at YAPC::Europe (Yet Another Perl Conference) in Riga, and just listened to the vision for Perl 5's future. Jesse Vincent is the Perl 5 Pumpking, essentially the project manager. His vision is to evolve Perl 5 into a modern language (he did not use those words, but that's the gist).
<br />
<br />Old Perl is 5.14, the current stable release. The idea is to introduce changes, like a fixed smartmatch, and deprecate the stuff that was preventing Perl 5 from moving forward because of the need for backward compatibility. Backward compatibility in the future will be provided by stating which version of Perl we should use - but still complete backward compatibility will not be guaranteed.
<br />
<br />By moving platform specific features out of the base and into modules (as in Python), the base gets to be small and easier to maintain, and maybe even faster. Deprecating features then removing them should have the same effect.
<br />
<br />What a shame this was not decided on ten years ago, still, better late than never. There will be issues with CPAN. Modules are usually only maintained for one version, the current stable one if you are lucky. Backward compatibility will be an issue with modules. The 'use version' directive (hopefully) will be lexical, so modules will be able to run on a later release (mostly, maybe). But of course they won't be able to run on an earlier release - but we have that issue right now. CPAN will need to search/tag/group modules by base release, that should not be hard, but it needs to be done before the issues arise at 5.16.
<br />
<br />So where does that leave Perl 6? Perl 6 is not dead, but rather it takes the pressure off it. Most people (me included) looked to Perl 6 for the future, and all the smart stuff that's in Python that I can't do in Perl. No longer. Perl 5 will move forward, alongside its sister language. The possibility of Perl 5 overtaking Perl 6 was not discussed.
<br />
<br />The times, they are a-changing.
<br />Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0tag:blogger.com,1999:blog-11940592.post-44516207547916030152011-04-07T10:18:00.005+01:002011-04-07T10:28:54.331+01:00Installing Python 3.2 Tkinter on LinuxA quick note for the install. Tkinter is not installed by default with Python 3.2 on Linux unless the tk development pack is installed. Where that is documented I could not say.<br><br>On Red Hat derived distributions this would be using (under user root):<br><span style="font-family:courier new;">yum install tk-devel </span><span style="font-family:courier new;"></span><br>Make sure you do this <em>before</em> building Python. It is required for<span style="font-family:courier new;"> idle</span>, which is named <span style="font-family:courier new;">idle3</span> (and that is not obvious either).Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0tag:blogger.com,1999:blog-11940592.post-63876717213851211712010-12-27T11:49:00.002+00:002010-12-27T12:01:58.133+00:00A Christmas present from the monksAfter what seems an age I finally got promotion to Prior, next stop (in 3000 XP points time) is Monsignor. My best node was <a href="http://www.perlmonks.com/?node_id=851683">Unexpected Output</a> with a score of (strangely) 42.<br /><br />It's weird, I get such a buzz from helping people with Perl it is almost adictive.Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0tag:blogger.com,1999:blog-11940592.post-76243072476436378412010-11-01T16:52:00.004+00:002010-11-01T17:08:59.958+00:00Python msvcrt.locking trials and tribulationsIt should be easy. All I wanted to do is to replace my C demos which use flock with a nice homely Python version on Windows. I wanted to stick with the standard library so I could just slot it into the existsing course (so I couldn't use the Locking module on PyPi).<br /><br />A simple demo to start with:<br />1. Process 1: lock a region, write a record, pause<br />2. Process 2: attempt to lock the same region<br />3. Allow Process 1 to release the lock<br />4. Process 2 continues<br /><br />Converting the code was easy (I thought), but no matter what I did I always found the region was locked for the second process - the lock was not being released.<br /><br />I blame the documentation, and possibly the implementation.<br /><br />What is not obvious is that the current file position must be reset to the original locking position before the lock gets released. The write (of course) advances the current file position, so by the time we do the unlock we are unlocking a region for which we don't have the lock in the first place! Wouldn't it be better if msvcrt.locking returned a lock object, saving the file position? Anyway, here is my completed demo:<br /><br /><pre><br />"""<br /> lock_w.py<br /><br /> Run two copies, each in its own terminal session.<br /> Allow one to write a number of records, then switch<br /> to the other showing that it blocks on the same record.<br /> Switch back and release the lock, and show that the<br /> blocked process proceeds.<br /><br /> Also run with lock_r to demonstrate interaction with<br /> read locks.<br /><br /> Default filename is rlock.dat<br /><br /> Clive Darke QA<br />"""<br /><br />import msvcrt<br />import os<br />import sys<br />import time<br />from datetime import datetime<br /><br />REC_LIM = 20<br /><br />if len(sys.argv) < 2:<br /> pFilename = "rlock.dat"<br />else:<br /> pFilename = sys.argv[1]<br /><br />fh = open(pFilename, "w")<br /><br /># Do only once<br />Record = dict()<br />Record['Pid'] = os.getpid()<br /><br />for i in range(REC_LIM):<br /> Record['Text'] = "Record number %02d" % i<br /> Record['Time'] = datetime.now().strftime("%H:%M:%S")<br /><br /> # Get the size of the area to be locked<br /> line = str(Record) + "\n"<br /> <br /> # Get the current start position<br /> start_pos = fh.tell() <br /> <br /> print "Getting lock for",Record['Text']<br /> msvcrt.locking(fh.fileno(), msvcrt.LK_RLCK, len(line)+1)<br /> fh.write(line) # This advances the current position<br /> <br /> raw_input("Written record %02d, clear the lock?" % i)<br /> <br /> # Save the end position<br /> end_pos = fh.tell()<br /> <br /> # Reset the current position before releasing the lock<br /> fh.seek(start_pos)<br /> msvcrt.locking(fh.fileno(), msvcrt.LK_UNLCK, len(line)+1)<br /> <br /> # Go back to the end of the written record<br /> fh.seek(end_pos)<br /> <br /> print <br /><br />fh.close()<br /><br /></pre>Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com1tag:blogger.com,1999:blog-11940592.post-3100196905427183702010-10-13T11:12:00.002+01:002010-10-13T12:12:31.674+01:00UpdateI try to keep these posts technical, but I have not blogged in a while so I'll break with tradition.<br /><br />I have been teaching a lot of Python. Mostly this has been Python 2, but some clients want Py3. Unfortunately not everyone understands how Open Source language releases work, and some internally assumed that no-one would want Python 2 training as soon as Python 3 was released.<br /><br />I have to confess that, having done a lot of Python recently, moving back to Perl is rather a drudge. Python is not perfect, but I now find the syntax of Perl unnecessarily fussy, and Perl 6 is even worse. I mean fussy in the sense that Perl code is to Python as a doily is to a beer-mat.<br /><br />Talking of Perl 6, with the release of Rakudo Star I thought (hoped) that we would get a flood of requests for courses, but we have not had one. Rakudo Star is, I guess, still too early, and I suppose early adopters are happy to learn it themselves.<br /><br />Meanwhile, roughly half of the delegates that I am teaching Python have come from Perl. There does not appear to be a consistent reason for this, and often it is not the practitioner's decision anyway. It appears to be the perception of where Perl is in the scheme of things. Its all about image and marketting. What can Perl do about that? Perl 6, but Python is catching up.<br /><br />Here is a simple example. One of the nice things about Perl is the way that lists can be used:<br /><pre><br />($one, $two, @fred) = qw(The quick brown fox);<br /></pre><br />Cannot do that in Python 2, but in Python 3:<br /><pre><br />one, two, *fred = ('The', 'quick', 'brown', 'fox')<br /></pre><br />The * indicates a greedy list (and you though Python didn't have sigils?). No qw() equivalent yet though, and Perl array and hash slices are still more powerful. Unless you know better...<br /><br />Now I learn that Civilisation V is using Lua as its scripting language because Python (used in Civ. IV) is too slow. Is Lua next on my list?Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com2tag:blogger.com,1999:blog-11940592.post-29475306348822193832010-08-05T08:46:00.004+01:002010-08-05T08:52:45.357+01:00Sony Vaio Windows 7 "No Internet access"This error has been driving me crazy. I got it when trying to connect to certain WiFi's, but not all. Surfing the web gave various solutions, none of which worked. I eventually cracked it by accident, so I'm posting it in the hope I might prevent someone else from going insane. It is far too late for me.<br /><br />Sony bundles all sorts of clever programs on its laptops. It's mostly concerned with multi-media, but one of them concerns us - VAIO Control Center (sic). So go to start/"All Programs" and select it. Now select "Network Connections", then "VAIO Smart Network".<br />Here be Dragons.<br /><br />VAIO Smart Network creates "profiles". Click on "Advanced", recognise that stupid dialogue displayed at start-up? Click "Settings" (bet you never thought of going there).<br />Now select "Profile Settings" from the left panel. Yes, I know this navigation is tortuous, but we are nearly there. From here you can edit one of your profiles, or create a new one. When a new profile is created from the desktop dialogue when it first connects, it defaults everything to the previous settings. It was carrying over a DNS setting I had from another connection, and that was preventing me from connecting. Select the "IP and DNS" tab and ensure that "Obtain an IP address automatically" and "Obtain DNS server address automatically" are both selected (it was the later which screwed me).<br /><br />The profile is selected by the name of the network. One thing that Sony engineers did not take into account in their design is a name collision. That is, two distinctly different networks having the same name. Many of our training centres have WiFi's, and they all are named the same, however they have different DNS addresses. That allowed me to connect to one training centre successfully, but then fail at all the others because the profile remembered the DNS address from the previous site. Going through the Microsoft Windows network settings did not detect where this was being held, because Sony were causing the problem not Microsoft (for once).<br /><br />Hope this helps.Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com32tag:blogger.com,1999:blog-11940592.post-66317266076567379492010-07-25T09:26:00.005+01:002010-08-02T10:58:53.658+01:00A view from EuroPython 2010I have just returned from EuroPython which was, like last year, in Birmingham, UK. This post will not go into too much technical detail, email if you need more. I'm guessing that the reader does not want to know how Python implements IEEE 754 floating point format (that's even mentioned in the Programming Foundations course, do try to keep up!). It requires about 4000 lines of C code to convert between float and text - no, you didn't want to know that.<br />Neither am I going to give a blow-by-blow account of each talk - they are available on the europython 2010 website.<br />There were just under 400 delegates this year - slightly down on last year. The organisers should not be concerned though: early indications from rival conference YAPC Europe (Yet Another Perl Conference) is that attendance is approximately half of last year.<br />Organisation was slightly better than 2009, a number of lessons appear to have been learnt. Talk streams were still patchy, but I guess that's inevitable.<br /><br /><br /><strong>Multiprocessing, the GIL, and threading</strong><br />The conference was opened by Russel Winder who impressed last year as well. The theme was very familiar to me, it told the story that Intel, and others, have been banging on about for a couple of years now - the fact that multi-core CPUs are here to stay and currently the only way to get the increase in speed beloved of developers. Russel went much further though, with my brain screaming ME TOO! Where Intel's solutions are distinctly small scale with core numbers in single figures, Russel spoke of much larger clusters (note the term). His contention was that current hardware solutions with cache are not scalable beyond 16, and neither are software solutions using threads. Message passing systems, like MPI, are a more likely future than threading systems like TBB or OpenMP. And no one in their right mind would be programming native threads. Here, here. A thousand times, here, here. All these lessons were learnt on mainframes in the 1970s: those that ignore history are destined to repeat it. It is a great shame that Russel's talk was rushed.<br /><br />That did not stop discussion in the conference about the vagaries of the Global Interpreter Lock (GIL), the much maligned excuse for not doing multithreading in Python. Hey guys: multithreading is hard, error prone, and not necessarily all that faster. Let's just accept that and move on. There will be a reworked GIL in Python 3.2, work which has come out of Google's "Unladen Swallow" project. It will be interesting to see how much that helps (?) and what excuses people will use to avoid multithreading in the future.<br /><br />I was browsing one of the book stalls, looking at an Advanced Python book when it hit me how much Python has moved forward in the past couple of years. The section on Multiprocessing mentioned an out-of-date module, not Subprocess and Multiprocessing. I dismissed the book as "out-of-date" even though the first edition was 2008 - QA's own Python courses have always covered those two modules. Then I realised that we have only had our own courses for about a year, although I have been tinkering with Python for (wow!) over ten years.<br /><br /><br /><strong>Python's progress</strong><br /><span class=" transl_class" id="4" title="Click to correct">Version </span>2.7 of Python was released on 4th July. OK, I'll update the QAPYTH2 course material as soon as someone gives me the time. (Actually many of the significant changes in 2.7 are back-ports from Python 3.1, so I can flitch the material from my Python 3 course. Just don't tell the boss). Python 2.7 is the last Python 2 major release. <span class=" transl_class" id="3" title="Click to correct">Version</span> 3.1 is now the only Python development stream, although 2.7 will continue to be maintained for around five years. When I started writing our own Python course material in late 2008 I was interrupted by the Python 3.0 release literally as I was writing the PowerPoint slides. I decided that a new course on Python 2 was daft, and switched to Python 3. It just so happened that I thought that Python 3 was a vast improvement for the language as well (don't cry for me, Perl 6). Six months later at EuroPython 2009 I realised I might have made a mistake, so I back-ported the course material to Python 2. I ended-up with two courses and "let the market decide". One year on and we have taught many more Python 3 courses than Python <span class=" transl_class" id="2" title="Click to correct">2</span>. and we are in a great position to move forward, unencumbered by legacy Python. Maybe it wasn't such a bad idea to target Python 3, but it was scary to be ahead of the curve.<br /><br />Python 3.2 should be out around the end of the year. One speaker gave January 2011 and another said "before the end of this year". Take your pick. Christmas? Python 3.3 was mentioned a few times, it should include Google's "Unladen Swallow" (yes Ian, it is from "Monty Python and the Holy Grail") with a target of a 5 times performance improvement (European or Asian?). No dates yet.<br /><br />My impression is that many developers still have not realised the benefits of Python 3, and have no plans to move over. They will.<br /><br />It looks like Python will overtake Visual Basic in the Tiobe stakes in the next few months (it passed Perl a couple of years ago). Are developers realising the benefits of Python? They are.<br /><br /><br /><strong>The cheese shop (the Python module repository: PyPi)</strong><br />(Recent course delegate: "but the Monty Python cheese shop had no cheese in it!". Me: "neither has PyPi")<br /><br />One of the main benefits of Perl (what?) is CPAN. Dave Cross (well-known Perlmonger) has said that this was the main reason for not moving to Python. Well Dave, Python just had its 10,000th module uploaded (round of applause). Actually I'm not so sure that is such a good thing, duplication and crud makes navigation difficult. Still, it's an indication of popularity and progress.<br /><br /><br />One of the main benefits of Python 2 against Python 3 is the cheese chop. Here too Python 3 is catching up. The module numbers are still only a few hundred, but there major modules are appearing all the time, for example NumPy (numerical processing) was just released for Python 3.<br /><br /><br /><strong>Wot I learnt</strong><br />Quite a lot, as always. I learnt the expression for such as <span style="font-family:courier new;">__path__</span> is "dunder path". Nice one. I also learnt about the importance of the new unittest module (that'll have to go in) and changes in the way import works for Python 3.2. I discovered how technicians lie about language comparisons to their managers to justify using a cool product (actually, I already knew that). Speakers were at times less than accurate with their comparisons against Perl.<br /><br />I learnt how HTML5 is going to make Microsoft Silverlight obsolete. Well the speaker did not actually say that, but I can dream.<br /><br />There were some fascinating statistics, like there are more people in India with access to a mobile 'phone than to a flush toilet (please don't take the comparison further). I would love to use that in a course if only I could find the derivation.<br /><br />The portability issues of using ActiveX are well-known to anyone using Microsoft's remote desktop (there are no portability issues - it's not portable), but apparently not to the South Korean government.<br /><br />The spec. for HTML1 was three pages, for HTML5 is 900. That's progress.<br /><br />Oracle had a stand at the conference, and they were giving out DVD's with developer's resource on for, let's see, Linux, PHP, Ruby, Python, and Oracle VM. Hummm, spot the missing language. Me: "What about Perl then?"; Oracle chappie: "oh, that's only used by system administrators for scripting". Go figure.<br /><br />And you can write cool games in Python, drive neat little robots, automate PowerPoint slides (I gotta get me some of that, and the robot stuff). And the Python Software Foundation are as cheerfully disorganised as everyone else. As with all conferences, it is always nice to have opinions confirmed, and to be able to say from time to time "actually, I knew that".<br /><br />Oh, and I saw a great example for "If then else for the lazy" in our UNIX fundamentals course:<br /><span style="font-family:courier new;">./configure && make</span><br /><br /><br /><strong>Finally...</strong><br />Python is self-assured, confident, and looking forward. The community is proud of its product, and itself. So they should be.Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0tag:blogger.com,1999:blog-11940592.post-37330744724102157552010-03-04T19:40:00.005+00:002010-10-13T13:58:45.991+01:00Python exercise - alternative solutionI promised to post an alternative solution to a Python exercise for an on-site today. For other interested parties, the code lists unused ports from /etc/services:<br /><pre><br />import sys<br /><br /># set the file name depending on the operating system<br />if sys.platform == 'win32':<br /> file = r'C:\WINDOWS\system32\drivers\etc\services'<br />else:<br /> file = '/etc/services'<br /><br /># Create an empty dictionary<br />ports = dict()<br /><br /># Iterate through the file, one line at a time<br />for line in open(file):<br /><br /> # Ignore lines starting with '#' and those containing only whitespace<br /> if line[0:1] != '#' and not line.isspace():<br /><br /> # Extract the second field (seperated by \s+)<br /> pp = line.split(None, 1)[1]<br /><br /> # Extract the port number from port/protocol<br /> port = pp.split ('/', 1)[0]<br /><br /> # Convert to int, then store as a dictionary key<br /> port = int(port)<br /> ports[port] = None<br /> <br /> # Give up after port 200<br /> if port > 200: break<br /><br /># Print any port numbers not present as a dictionary key<br />for num in xrange(1,201):<br /> if not num in ports:<br /> print "Unused port", num<br /></pre><br /><br /><br /><br /><br /><br /><br />Here is a smaller solution using Regular Expressions:<br /><pre><br />import sys,re<br /><br />file = r'C:\WINDOWS\system32\drivers\etc\services' \<br /> if sys.platform == 'win32' else '/etc/services'<br /><br />found = set()<br />for line in open(file): <br /> m = re.search(r'^[^#].*\s(\d+)/(tcpudp)\s',line)<br /> if m:<br /> port = int(m.groups()[0])<br /> if port > 200: break<br /> found.add(port)<br /> <br />print set(range(1,201)) - found <br /></pre>Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com1tag:blogger.com,1999:blog-11940592.post-42974909641000650612010-01-29T09:40:00.004+00:002010-01-29T10:05:14.365+00:00On the takeover of Sun by OracleI guess I might as well comment - everyone else and his dog is doing so. A collegue said "A sad, sad day for FOSS". I know what he means, but actually I disagree. Take-overs and mergers will always happen. <br /><br />It is occasions like this that demonstrate the power of FOSS. FOSS gives us choice, and we can choose to stay with MySQL or migrate to PostgreSQL, which all the smart people were using anyway. IMHO, technically PostgreSQL knocks MySQL into a cocked hat, and the Oracle takeover of MySQL could give PostgreSQL the increase in popularity it deserves. Other databases are available.<br /> <br />Sun blew hot and cold on FOSS anyway, I could never figure out what their policy was. Java probably has too much momentum of its own for Oracle to screw it up, although if anyone can.... Never underestimate the ability of corporates to kill things off by hubris (I used to work for Computer Associates).<br /> <br />There will always be other languages, other databases, other operating systems.<br /><br />Other hardware?<br /><br />Back in the 1980s a collegue (Haydn Moston - are you still around?) told me that CISC (Complex Instruction Set Computer) chips could never last beyond 2000, and RISC (Reduced Instruction Set Computer) chips were the only way to go. The i386 familiy is CISC, and Sun Sparc is the most well-known RISC. <br /><br />Intel are running out of steam and having to use multi-core, ten years out was actually not bad Haydn. I'm not sure that the switch to multi-core is connected specifically with CISC, but the possible loss of RISC machines is my biggest worry with the Oracle takeover. The number of mainstream instruction sets out there is disapointingly small. How I hate monocultures.Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com1tag:blogger.com,1999:blog-11940592.post-33280932108463283092010-01-07T14:53:00.006+00:002010-04-10T17:27:19.730+01:00File notification events on WindowsOne of my most popular posts is example code for Inotify on Linux. I have also been asked for similar code for Windows, so yer tis:<br /><br />There are two different interfaces available on Win32, I prefer ReadDirectoryChangesW because it is easier to control:<br /><br /><pre><br /><br />// ------------------------------------------------------------------<br />// Clive Darke QA Training<br />// ReadDirectoryChangesW example<br />// ------------------------------------------------------------------<br /><br />#define _WIN32_WINNT 0x0400 // <<<<<<<<><br />#include <iostream><br />#include <windows.h><br /><br />void DisplayLastError( LPSTR lpszText );<br /><br />// ------------------------------------------------------------------<br /><br />int main ( int argc, char *argv[] )<br />{<br /> HANDLE hDir;<br /> DWORD dwReturned;<br /> BOOL bResult;<br /> FILE_NOTIFY_INFORMATION *pNotify;<br /> <br /> if ( argc < 2 )<br /> {<br /> cerr << "You must supply a directory name" << endl;<br /> return 1;<br /> }<br /><br /> // Note FILE_FLAG_BACKUP_SEMANTICS, which is the strange<br /> // attribute required to get a handle to a directory. <br /><br /> hDir = CreateFile (<br /> argv[1], // pointer to the file name<br /> FILE_LIST_DIRECTORY, // access (read-write) mode<br /> FILE_SHARE_READ|FILE_SHARE_DELETE, // share mode<br /> NULL, // security descriptor<br /> OPEN_EXISTING, // how to create<br /> FILE_FLAG_BACKUP_SEMANTICS, // file attributes<br /> NULL // file with attributes to copy<br /> );<br /><br /> char Buffer[MAX_PATH] = {0};<br /><br /> while (TRUE )<br /> {<br /> char szAction[42];<br /> char szFilename[MAX_PATH];<br /><br /> bResult = ReadDirectoryChangesW (hDir, &Buffer, sizeof(Buffer),<br /> TRUE, FILE_NOTIFY_CHANGE_FILE_NAME, &dwReturned, NULL, NULL);<br /> <br /> if ( !bResult )<br /> break;<br /> <br /> pNotify = (FILE_NOTIFY_INFORMATION *) Buffer;<br /><br /> switch (pNotify->Action)<br /> {<br /> case FILE_ACTION_ADDED : {<br /> strcpy (szAction, "added");<br /> break;<br /> case FILE_ACTION_REMOVED :<br /> strcpy (szAction, "removed");<br /> break;<br /> case FILE_ACTION_MODIFIED :<br /> strcpy (szAction, "modified");<br /> break;<br /> case FILE_ACTION_RENAMED_OLD_NAME :<br /> strcpy (szAction, "renamed");<br /> break;<br /> case FILE_ACTION_RENAMED_NEW_NAME :<br /> strcpy (szAction, "renamed");<br /> break;<br /> default:<br /> strcpy (szAction, "Unknown action");<br /> }<br /><br /> wcstombs( szFilename, pNotify->FileName, MAX_PATH);<br /> cout << "File " << dwerror =" GetLastError();" lpmessagebuffer =" NULL;"><br /></pre>Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0tag:blogger.com,1999:blog-11940592.post-75338056051985258442009-12-08T11:19:00.001+00:002009-12-08T11:21:37.950+00:00ASSOC on Windows 7A while back I posted some suggestions for .bat files to run cmd.exe sessions for Python. I use these a lot for PHP and Perl as well. However there is an issue with them on Windows 7. "Out of the box", the ASSOC command gives "Access is denied", even though the user is an Administrator.<br /><br />My first thought was to turn off the dreaded User Account Control settings, but that changed nothing. The eventual solution was to go into regedit and grant my user full control on registry key HKEY_CLASSES_ROOT, which is where the file associations are held. Fortunately Windows security for keys (and directories) is based on inheritence, and so keys within that hive will have the same permissions by default.<br /><br />I can see why file association sould be seen as a security risk, so why don't Microsoft finally admit that the filename is not a good way to identify the file type, and use magic numbers like everyone else? Microsoft Office files, and other Microsoft proprietry formats, all contain magic numbers, and if you place one of these files onto Linux then it identifies them correctly without the need for file extensions.<br /><br />UNIX's #! line for script files is a consequence of the magic number system. It is simple and incredibly flexible. When C# was developed the format of the .exe files (PE) had to change to support it. If they implemented #! then Microsoft would not have had that problem.<br /><br />When I wrote the yash shell I simply read the commands from STDIN. On UNIX I needed no special code to support yash scripts - just the #! line. But on Windows I had to associate a file extension.<br /><br />If I want to run Python 2 and Python 3 on the same machine on UNIX or Linux then I just have different #! lines at the top of the script. On Windows, because they have the same file extension, I have to redo the file associations every time I want to run one of them. Dire.Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0tag:blogger.com,1999:blog-11940592.post-25644271226256417892009-08-11T15:56:00.003+01:002010-03-16T13:42:59.551+00:00A winning feature of Perl 6So what features do Python programmers love to say about their code? First off they like to crow about how clean their code looks without sigils, those nasty $, @, %, &, * that Perl programmers have to put up with. Ah, but that means you cannot do any interpolation in Python strings, because Python does not know the difference between bare text and a variable name.
<br />
<br />Anyway, Python uses _ and __ (double underscore) to change the behaviour of variables, if these are not sigils then what is? Python uses an @ to prefix a decorator. Sounds like a sigil to me.
<br />
<br />Perl 6 mandates that variables are declared using 'my' (or some other scope indicator), but Python does not require that. This means that in Python you cannot declare a variable for scope inside a local block, like an 'if' or loop.
<br />
<br />Python does not use braces to delimit a block. Nice, unless you need to declare an empty block, so you need a 'pass' statement (reminds me of NEXT-SENTENCE).
<br />
<br />Most of that is trivial to be honest, none is important enough to make a strong case either way.
<br />
<br />So what is? Threading.
<br />
<br />A recuring theme at the EuroPython conference was the fact that CPython is not truely multi-threaded because of the GIL (Global Interpreter Lock). Ruby has the Global VM Lock. There are various implementations which bypass this issue, Jython and Iron Python do not have a problem with multi-threading but both are several versions behind and I won't be holding my breath for a Python 3 version. Google is apparently working on a CPython version which eliminates the GIL, but again that is some way off.
<br />
<br />In a classic "sour grapes", the Python community say that threading is too difficult anyway, so use processes instead. Very few people at the conference were swallowing that line.
<br />
<br />Multithreading should be a selling point of Perl 6 and Parrot, but unfortunately is not yet complete and will not be in Rakudo Star. That is a great pity. Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0tag:blogger.com,1999:blog-11940592.post-69502448406600128382009-08-05T05:10:00.002+01:002009-08-11T16:36:47.950+01:00Perl 6 gets a release date!Big news from the current Perl conference YAPC Europe 2009 here in Lisbon is that Rakudo * (star) will be released in spring 2010. Rakudo is Perl 6 on Parrot, which is an equivalent to the JVM or CLI.<br /><br />Is that an official Perl 6 release? Well, it is the nearest to official that you will see. The Perl 6 team are keen to point out that there will be no such thing as a official version, just different implementations. I guess they are looking at Python for their inspiration there.<br /><br />This could be make-or-break for Perl. It shouldn't be, Perl 5 will continue no matter what, but the new stuff is what will make Perl cool again in the eyes of the fashion conscious. We all know that fashion is fickle, illogical, and driven more by hormones than intellect. We also know how important it is in decision making. Much as it would like to, the Perl community cannot pretend that Perl 6/Rakudo is too far away to worry about. It is six months away.<br /><br />My worry is that expectations will be high - the release will be incomplete and they are very open about that. The Rakudo "pumpking" (project manager), Patrick Michaud, sees this as a test release to encourage people to write applications to find bugs and feed back into the development. Trouble is that companies have been burned too often in the past to invest in applications on an incomplete and possibly flaky platform. <br /><br />The dilemma is that if something is not released soon then Perl will have missed the boat - which is already upping anchor and about to steam away.<br /><br /><strong>Update: </strong> read the comments from <strong>Pm</strong> - they are very good.Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com4tag:blogger.com,1999:blog-11940592.post-6904234156301361832009-06-24T14:14:00.002+01:002009-06-24T14:19:54.174+01:00Perl 6 Questions and further informationThis is following a recent Perl 6 seminar I gave.<br /><br />Perl 5 style is still supported.<br /><br />Named parameters go into %_<br />Unnamed parameters go into @_<br /><br />Subroutines can be predefined with:<br /><pre>sub foo {…}</pre><br />… is known as the "yadayadayada" operator.<br /><br />Lexically scoped subroutines:<br /><pre>my sub mysub () is context { …}</pre><br /><br /><br />Optional parameters: ? now comes after the parameter name.<br /><pre>sub mysub ($value, @harry, %ash, $opt?) {…} </pre><br /><br />Named parameters: prefix is now : , instead of + and are optional unless they have a ! suffix.<br /><pre>sub named ($value, :@harry!, :%ash, :$other) {…}<br /><br />...<br /><br />named(42, other = 'fred', harry = @files);<br /></pre><br /><br />Slurpy parameters, like perl 5 lists and arrays, are prefixed *<br /><br /><strong>C style prototypes<br /></strong>Are not supported. It appears that when a subroutine is called before it is defined the parameter resolution still works correctly. However a runtime error is reported, not a compile time error (as it would with Perl 5 prototypes).<br /><br /><strong>Dynamic types</strong><br />These are called "traits". See Synopsis 12: Objects.<br /><br /><strong>Zip operator</strong><br />Has changed, it is now 'Z'. Behaviour with different length arrays has also changed between pugs and Rakudo, the result uses the shortest:<br /><br /><pre><br />my @array = qw(The quick brown fox);<br />my @brray = qw(Now is the time for all);<br /><br />my @all = @array Z @brray;<br />say @all.elems();<br />say @all[];<br /><br />@all = @brray Z @array;<br />say @all.elems();<br />say @all[];<br /></pre><br />Gives:<br /><br /><pre><br />8<br />TheNowquickisbrownthefoxtime<br />8<br />NowTheisquickthebrowntimefox<br /><br /></pre><br /><br /><br /><strong>Important further change</strong><br /><br />When testing these changes in Rakudo I ran up against a change I had almost forgotten. When calling a subroutine, no whitespace is allowed between the sub name and the open parentheses. That slowed me up until I remembered. A backward step IMHO.Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0tag:blogger.com,1999:blog-11940592.post-63851533911628924662009-06-23T10:09:00.002+01:002009-06-23T10:10:55.324+01:00MySQL VARCHAR limitsA delegate recently queried the size limit on a VARCHAR field in MySQL. Our course documentation states a maximum of 255 characters: that is correct. For larger fields use TEXT.Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0tag:blogger.com,1999:blog-11940592.post-61356887472202710222009-06-17T10:57:00.002+01:002009-06-17T11:03:57.688+01:00Perl 6 QuestionsI recently gave a short talk on Perl 6, and a number of questions arose. I will answer those questions as soon as I can, but unfortunately other fires need fighting right now.<br /><br />One of the reasons for the delay is that the syntax for subroutines for Perl 6 has changed again, and in fact my slides are now out of date. That also means that pugs and Rakudo are incompatible in a major way - they have always been out of step to some extent.Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0tag:blogger.com,1999:blog-11940592.post-62045323680915177452009-04-24T08:24:00.004+01:002009-05-05T10:01:52.337+01:00Moving into the light?A long time since my last post. Aside from bread-and-butter stuff, my activity has been on Python. There have been various upheavals in the business, and we aquired a Python 2 course. It wasn't too hot, and needed updating. Around the end of 2008 Python 3 was released, so I decided it would make sense to create a new course based on Python 3.<br /><br />That is almost finished, but it turns out that it might have been premature, since no one seems to be in a particular hurry to move to Py3. That's a shame, because I am quite impressed. There have been issues, performance has been one of them (although I can't say I noticed any problems). In June 2009 a new version, 3.1, should be released, and that has a number of new features. So, having done all that work on a new course, I will have to hack it again come June. That's the problem with the bleading edge.<br /><br />I have read of Python's ease of use, and never really believed it. After all, there are so many vested interests and religious wars that it is difficult to get a balanced view. I have to say that, after a little practice, Python is very easy to pickup. I have found myself <em>just coding</em>. The number of times I have to look something up gets less and less. I'm not sure I am fluent yet, but I'm close, and I reckon that is the fastest I have ever picked up another language.<br /><br />Will Perl 6 be as easy?Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0tag:blogger.com,1999:blog-11940592.post-81549818546795216602009-02-07T16:03:00.005+00:002009-02-07T16:28:27.290+00:00App::sh2p Version 0.05 now on CPANI always knew this would be a long project, although I'm heartened as to how far it has gone. It appears to have been useful to some people. It is a feature of Open Source that the only time I can be really sure it is being used is when I get bug reports - which could be rather off-putting to anyone without the skin of a rhino.<br /><br />This latest release, version 0.05, fixes quite a few bugs, and adds support for things like trap (signal handling). In my defence, most of the bugs are missing support for certain features, and it is still in beta. I still have not got pipelines sorted out (that is a huge job), or $?, but they will come eventually. I only get to work on this in odd moments, evenings and so on, and it is difficult to find the time for development stints of more that two hours. I could really do with spending a couple of solid days on it.<br /><br />One problem I had was in inserting a marker into a string which would indicate a token delimiter. Everything I tried clashed in someway with a real data item. I solved it eventually by using a gash reference, but I can't help thinking there must be a more elegant solution.<br /><br />I have also started using OO techniques more - this product really does benefit from that. <br /><br />I am now using the profiler and Coverage for my tests, but have not sorted out a good way to integrate my command-line tests. The "Perl Testing" book has ideas, but they require CPAN modules and I would rather not increase the number of dependencies. If you download this program and are dismayed at the tests (I basically check the thing will load) then be assured that I have a test suite of over 90 scripts, just one of which is my regression test script which currently has 55 test paragraphs within it. Each time I find a bug or add a feature I add a test, just like I should.Clivehttp://www.blogger.com/profile/06031966607543564115noreply@blogger.com0