Shock! Horror!! A delegate ate my machine!

OK, he didn't actually eat it, but he did exploit a "feature" of CentOS I was unaware of. The halt, reboot, and poweroff programs can be called from an ordinary user if that user is logged on to the console at the time. Our classroom users have very simple passwords, but the point is that these can be called from a non-root user.

I found a fix (with thanks to Mr. Google) but I'm sure there is probably a better one. Right now this will do, since it appears to work, but I'll try and find an alternative.

In /etc/security/console.apps there is a config file for the programs mentioned (and others). Set each one to something like (for example):

USER=nobody
PROGRAM=/sbin/poweroff
SESSION=true

Of course having better passswords would help...

Function redirection

I just came across this:

function myfunc {  

    echo 'Hollow world' 

} > outfile

Works with Bourne style functions as well, and is in the POSIX standard. It should not be a surprise, since it just follows the same standard as 'if' and loops, but I have never seen it before. It certainly is not our shell course (yet). Seen it before?

Here is a neat trick:

outfile=one.out

function myfunc {
    echo 'Hollow world'
} > $outfile

outfile=two.out
myfunc

Which file gets written to, one.out or two.out? Interestingly it is two.out. In other words the variable is resolved at runtime, not function declaration time.

App::sh2p - shell to perl converter

Just uploaded to CPAN, this has been a pet project of mine for a while (see perlmonks here ). It is currently only a alpha release, and you know what that means.

Converting UNIX shell scripts creates problems and my script (with associated modules) do not answer them all. The syntax of shell programs is very loose, almost anarchic. This is not a criticism of shells - they have a dual role to play in providing a usable command-line interface, as well as acting as a programming language. I target the POSIX shell, but there are many variations, not least those created by Sun. The Korn shell has moved on considerably from ksh88, and the syntax of the latest versions is complex. I make no attempt at the more esoteric parts of ksh93 syntax, expansions and extended globbing. There are annoying inconsistencies, fore example, what is the difference between:

[ $x -a $b ]

and

[[ $x -a $b ]]

I shall leave the reader to ponder (hint: -a has a different meaning to the Bourne-shell [ and the Korn shell [[).

As you might gather, this is not a simple syntax conversion. Take $? for example: that can be applied to any shell command you like and at any point in the program. Not many languages work like that, certainly Perl does not. Fortunately not many people use $? in shell scripts for anything other than testing the result of a child process - not unlike its use in Perl (there is a difference in the detail).

Right now App::sh2p attempts to remove some of the leg-work from conversion, and I hope t is useful.

Flip-flop operator

I have been using the flip-flop operator in Perl for ages, equating it with sed and awk line addressing. However a question just came up that asked how we process from a specific line until end-of-file. In sed and awk the last line of the file is denoted by $, but that does not work in Perl for obvious syntax reasons.

I had to think about that: the solution is to use eof(). So, to list a file from line 20 we can use:

while (<>)
{
   print if 20..eof 
}  

Promotion!

When I signed on to Perlmonks this morning I got:

You have 24 votes left today.
You gained 16 experience points. They really complement your eyes.
Your new level imbues you with super-powers! Check out Level 13: Curate for the details!
You have 991 points until level 14 - Priest.

The trials of a CPAN author

As mentioned previously I have a module called Win32::EnvProcess. The CPAN people (modules@cpan.org) don't like that name, although I'm not sure why. Changing a name is not clearly defined, and I don't get answers to email. Hey ho, it doesn't stop Win32::EnvProcess from being found and downloaded.

And that's the trouble with being a developer – people will actually try and use the stuff. So I get queries, and I get bug reports.

Generally the email queries are sensible, and I have had a few over the past week. First I, or rather a user, had an issue with Microsoft Visual Studio 2005, which is a version I haven't got around to using. It seems that good ol' Microsoft decided to fix the "DLL hell" problem by introducing a 'manifest' file into its link/load phase. So now we get "manifest hell" instead - the issues are well documented on the web. Of course the Perl MakeMaker does not generate a manifest file for its DLLs, and so my module wouldn't load and run. After some blind alleys and investigation I found a patch for MakeMaker, via perlmonks which I passed on. Of course that meant I had to document the thing as well. Meanwhile I found a small memory leak. OK, so that was worth it.

Then I got a bug report saying that the installation failed if the user did not have a C/C++ compiler. OK, I would have thought that if the product included an XS file, and the documentation showed compilation steps, then that would have been obvious. Clearly not, so more lines to the doc.

A user sent a "it would be great if…" feature request. "Good idea", I thought. "Simple patch", I thought. Only one of those thoughts was correct. Well you see it all worked on my Windows machines, it's just that it failed everywhere else. It wasn't until one of the cpan testers sent me a copy of his environment block that I cracked it. Their environment blocks were bigger than a hard limit I set – size matters. I hate hard limits and try and avoid them, but in this case I have to set one to a file mapping object. It's just that I was not handling the overflow case correctly.

There is nothing like a sustained bout of debugging to improve the diagnostics available in code. And of course I found another (unrelated) buglett as well – a possible race condition.

Then another bug report, this time someone is trying to compile the module under cygwin. Cygwin is a UNIX emulation environment, and this module is called Win32::EnvProcess – the clue is in the name. I check the OS name in the tests, but the thing won't compile and link anywhere except Windows. Of course not, RTFM. Alright, so Cygwin runs under Windows but it doesn't give access to the Windows kernel, it pretends it is UNIX.

Don't get me started about Strawberry Perl, just don't. Alright then. Strawberry Perl is a laudable package of open source products to enable you to install Perl without any dreaded Micro$osft licences. Good, great stuff, more power to them. Except that the Perl installation tools, like MakeMaker, doesn't really support it. So more magic in my Makefile.PL, already burdened by this stupid manifest issue. UNIX? Pah, that's easy.

Version 0.05 of Win32::EnvProcess has just been uploaded. God bless her, and all who sail in her.

Win32::StreamNames - new version!

My CPAN module Win32::StreamNames had a bug - I know, unbelieveable. Happens to the best of us. The symptoms were that empty data streams were not returned. It is now fixed in version 1.03 which I just uploaded.

The bug was an unexpected (and undocumented) behaviour of the Win32 API BackupRead when it finds an empty stream, and I discovered the fix just by experimentation. Good old Microsoft, guarenteed to keep life interesting.

Thanks to Frederic Medico for reporting the error.

Yet another time calculation in Perl

This is specifically for a delegate I am teaching right now. If I understand the requirements correctly, we hae two times in HH:MM:SS format and we wish to calculate the difference. I know, what if the times are on different days? No answer was given to that one, so I have assumed they are on the same day (today, to be exact).
This will probably change because the specification is a bit "wooly".

#!/usr/bin/perl

use strict;
use warnings;
use Time::Local;

sub str2time
{
   my $in = shift;
   my ($hour, $min, $sec) = split (/:/, $in);
   my ($mday,$mon,$year) = (localtime(time))[3,4,5];
   my $retn;
 
   # timelocal throws an exception on an invalid date/time
   eval {
      $retn = timelocal($sec,$min,$hour,$mday,$mon,$year);
   };
 
   return $retn;   # undef on error
}

print "Please enter the first time in HH:MM:SS format: ";
my $intime = <STDIN>;
chomp $intime;

my $time1 = str2time($intime);

die "Invalid time: $intime\n" if !defined $time1;
  
print "Please enter the 2nd   time in HH:MM:SS format: ";
$intime = <STDIN>;
chomp $intime;

my $time2 = str2time($intime);

die "Invalid time: $intime\n" if !defined $time1;


my $diff;
if ( $time1 > $time2 ) {
    $diff = $time1 - $time2
}
else {
    $diff = $time2 - $time1
}

my $mins  = int($diff / 60);
my $hours = int($mins / 60);
$mins     = $mins - ($hours * 60);
my $secs  = $diff - ($mins  * 60);

printf "Difference: %02d:%02d:%02d\n", $hours,$mins,$secs;

Away from the dark side - Python

Our Python guy is off sick, so I've spent a week hacking the language. I had looked at Python a few years ago, but this time its personal.

So these are the impressions of a novice - and someone coming from Perl at that. I will probably come-up with justifications and solutions for the shortfalls as I learn, they are probably a mark or my ignorance rather than anything wrong with Python. Having said that, I was surprised how many things are missing.

The good
1. Forced code formatting
2. No stupid sigils
3. OO built-in
4. Default variables in a function are not global
5. Documentation
6. OS specifics are removed from the core
7. Excellent string handling
8. Intuitive file IO
9. Method calls on built-in types instead of weird system variables
10. No GOTO

The bad (IMHO)
1. No constants
Even True and False can be altered!
2. No scoping within loops and 'if' statements
All variables outside functions are globals
3. No interpolation - an effect of the lack of sigils I guess
4. Comparisons between numbers and strings give unexpected results, and NO WARNINGS! A poor and surprising feature
5. I know ++, --, +=, etc. are arguably poor coding, but I missed them.

Still, 10 good and only 5 bad.

Win32::EnvProcess

YAPM (Yet Another Perl Module) which I just uploaded to PAUSE.

This module enables the user to alter or query an unrelated process's
environment variables.

Windows allows a process with sufficient privilege to run code in another
process by attaching a DLL. This is known as "DLL injection", and is used here.

SYNOPSIS

use Win32::EnvProcess qw(:all);

use Win32::EnvProcess qw(SetEnvProcess);
my $result = SetEnvProcess($pid, env_var_name, [value], ...);

use Win32::EnvProcess qw(GetEnvProcess);
my @values = GetEnvProcess($pid, env_var_name, [...]);

use Win32::EnvProcess qw(DelEnvProcess);
my $result = DelEnvProcess($pid, env_var_name, [...]);

use Win32::EnvProcess qw(GetPids);
my @pids = GetPids($exe_name);

This is another case where a question on perlmonks generated the interest. How do I get the child to create/alter an environment variables in the parent? Had this been on UNIX then the answer would be simple - you can't without having co-operating proceses. On Windows however DLL injection makes this possible.

Enjoy!

I have done a brief investigation on how this might be achieved on Linux, but I don't think it can fly. The API ptrace(2) is a start, but the problem is in creating a thread in the host. Addresses in the host can be changed, but the environment block is not at a fixed location so far as I know, and without a symbol table I'm not sure how you would find it.

Win32:IdentifyFile

I just uploaded my fourth Perl CPAN module to PAUSE (the Perl Author's Upload Server).

This one was written because of a conversation on perlmonks Chatterbox. A module author was bemoaning the fact that Windows files do not have an inode number. I waded in to say that they do, kinda. However it's not easy to use, and requires a C API to be called. Hence the module, written using XS.

There are two functions, IdentifyFile() and CloseIdentifyFile().

IdentifyFile() returns 3 components to uniquely identify the file or directory. In list context these are returned as a 3 item list, in scalar context they are joined together as a single string using ':'. It is therefore simple to compare two identities in perl, using 'eq'.

Just getting this information could result in a race condition. The file could be deleted, possibly by another process, between getting the information and using it in a test. Worse, another file might be created with the same file index meanwhile. To prevent this scenario, files (or directories) are opened internally by IdentifyFile(), and kept open until CloseIdentifyFile() is called (files are not physically deleted until all open file handles are closed).

I wonder how many people think of that when using an inode number on UNIX?

UNIX command equivalents in Perl

I realise I said I would post a new update in an ealier post. Well this is not it!
For the very latest list see my perlmonks post: UNIX command equivalents in Perl

First impressions in India

On the road again, or rather "10 cramped hours in cattle class".

My hotel is a Marriott, not everyone's cup of tea, but I think they are excellent. In my previous job we always stayed in them, nowadays I rarely get the chance; usually the hotel is um, economic. Anyway, the luxury of the hotel is in stark contrast with the surroundings. Quite a culture shock, it is like living in a travelogue.

Stereotypical taxi from the airport, including the dangly bits on the mirror. Streets being brushed by doubled-up women holding rushes, and this at 1:30 AM. The reflective safety jackets were particularly incongruous. And yes, there was even a cow in the road.

The guards, sniffer-dog, and metal detectors at the hotel entrance were a bit of a shock. The helpful and polite staff are overwhelming. I think I have said "thank-you" more than any other phrase since I got here.

Walking out of the hotel on Sunday morning I was surrounded by tricycle drivers wanting to take me for a tour. When I politely declined they disappeared, and I was not bothered again. Just outside the hotel the poverty is heart-rending, yet the busy road ignores them. I have never seen anyone riding side-saddle on a motorbike while wearing a sari before.

(Windows) What is the difference between a thread's HANDLE and its ID?

There seems to be some doubt amoung my peers over the use of a Thread ID, referred to here as a TID.

The most obvious difference between a HANDLE and a TID is that a HANDLE is specific to a process, whereas a TID is system wide.

A HANDLE is actually an unsigned int (whereas a UNIX file descriptor is a signed int) and is an index into the process's Handle Table, which is maintained by kernel.

A TID is a type of Client ID, the other type of Client ID being a PID (Process ID). TIDs and PIDs are generated in the same namespace and used by the kernel to identify these objects system wide.

So, why would you need the TID? You don't. At least, not often. A TID may be passed between processes, and another process can then get a HANDLE to that thread using the OpenThread() API.

Yes, you can manipulate threads in other processes - the caveat being that security may stop you, particularly in Windows Vista. Once you have a HANDLE to another thread you can do all those wonderful things, Suspend, Resume, Wait, GetExitCode, as you can if the thread is in the same process. One of the useful APIs in this context is PostThreadMessage(). This sentence is mis-leading, PostThreadMessage takes a TID, see comment from Hibou57 below.

BTW: make sure that HANDLE is closed, the thread cannot truly die until all HANDLEs to it are closed - that is how GetExitCodeThread() works. And you thought zombies only occurred on UNIX?

It is possible to use DuplicateHandle() to get a handle opened by another process, but that is far more painful than the MSDN implies.