samedi 29 décembre 2012

Loterie 4-1-9

A quelques heures de 2013, je ne résiste pas au plaisir de partager ce morceau d'anthologie que je viens de recevoir à l'instant.Malgré les différents filtres, on en reçoit tous un peu, de qualité plus ou moins variable. Encore il y a quelques années, je m'amusait à les collectionner dans une boite dédiée de mon client courrier, avec l'objectif initial de constituer un corpus pour un travail futur. Devant le peu d'originalité déployée (et aussi la quantité reçue), ça.m'a passé, et en général, rien qu'au titre, j'ai déjà pressé la touche "Suppr"...

Mais pour celui-ci, ça vaut le coup d'oeil. Le texte associé est plutôt banal (je ne vous le recopie pas, vous l'avez tous déjà lu ailleurs), mais l'image envoyée montre que le gars y a mis tout son coeur! Texte en anglais, photo de Bill Gates, "joli" dégradé de couleur sur le titre, et puis surtout, presque aucune faute d'orthographe, je n'en ai vu qu'une (je vous laisse la chercher...)


Ca serait presque drôle, on peut se dire qu'un gamin (nigérian ?) a pu acquérir quelques compétences en infographie grâce à ça (bon, en même temps, il n'y en pas pour des heures...), mais en même temps, ça laisse rêveur. Y a-t-il réellement, fin 2012, des gens qui peuvent gober un truc pareil ? Si oui, ça témoigne d'un sacré problème. Vaste sujet...

lundi 23 avril 2012

Impression de planches d'étiquette avec Latex (publipostage)

Je m'occupe d'une association avec laquelle nous faisons de temps en temps du mailing "classique" (par la poste!) en générant des planches d'étiquettes autocollantes à partir d'un fichier de contacts maintenu sous la forme d'un fichier "tableur".
Pendant longtemps, je me suis acharné, (voire épuisé!) avec les "fonctionnalités" de publipostage des suites bureautiques classiques.
Cette fonctionalité, pourtant à priori basique, est en effet difficile à manier, du moins sur les versions que j'ai pu essayer (Office 2003 et OOo 3.3). On passe des heures à créer un modèle, à cliquer, à en enregistrer les versions successives, à identifier le format parmi les formats prédéfinis, à cliquer, à ouvrir la base de données (qu'il faut aller rechercher dans l'arborescence à chaque fois...), à fusionner, à recommencer parce qu'on a un message disant "impossible de trouver la source de données" ou quelque chose du genre, à recliquer, à re-naviguer jusqu'au dossier, à cliquer encore, à re-générer, à essayer autre chose, à re-naviquer, à re, à re, etc.
L'enfer! Pour quelque chose qui au final, peut fonctionner, mais surtout a un comportement erratique: un coup, c'est bon, un coup, c'est pas bon....

Bref. Insupportable.

A coté de ça, on trouve aussi des logiciels, libres ou propriétaires, qui vous proposent de faire peu ou prou la même chose. Mais bon, encore un bidule à installer, qui ne va pas forcémment être exactement adapté à ce que vous voulez...

Je refléchissais depuis quelque temps sur la possibilité d'une solution automatisée 'en 1 clic', utilisant des solutions éprouvées, libres et multiplateforme, mais faute de temps, j'avais laissé ça un peu de coté. Etant utilisateur quotididien de Latex, j'ai presque par hasard recherché s'il n'y avait pas un package qui m'aiderait dans cette tâche. Et là, Bingo!
Premier hit: le package "labels", qui fait... tout!
Pour la faire courte, et pour les familiers de Latex, il suffit du petit source suivant. Pour les autres, c'est peut-être une excellente occasion de se mettre à Latex ?

\documentclass{article}
\usepackage{labels}

\begin{document}
\begin{labels}
\input names.dat
\end{labels}
\end{document}

Et le tour est joué: après compilation, vous disposez d'un pdf avec les étiquettes, à condition que vous fournissiez dans le dossier courant le fichier names.dat contenant les noms et adresses au bon format (voir ci-dessous). Evidemment, en général, on devra personnaliser un peu, ajouter babel pour l'alphabet occidental, le nombre de lignes et de colonnes, etc. On peut évidemment régler les marges et autres espacements avec les commandes adéquates, la doc est très complète là dessus, et l'archive de téléchargement contient en bonus une multitude d'exemples.

# my_labels.tex

\documentclass[a4paper,12pt]{article}
\usepackage[french]{babel}
\usepackage[utf8]{inputenc}% ou iso8859-15 en général sur windows
\usepackage{labels}

\LabelCols=2
\LabelRows=7

\begin{document}
\bfseries
\begin{labels}
\input names.dat
\end{labels}
\end{document}

Pour le format des données, ce package n'attend pas de format particulier: il se contente d'imprimer sur les étiquettes le texte fourni, ligne par ligne, avec comme séparateur d'étiquette un simple saut de ligne. Comme par exemple (tiré de la doc):

Professor R. Bercov, Chair
Department of Mathematics
University of Alberta
Edmonton, Alberta
Canada T6G 2G1

Chair of the Search Committee
Department of Mathematics
and Statistics
University of Regina
Regina, Saskatchewan
Canada S4S 0A2

...

Il faudra donc commencer par exporter les données au format csv, puis procéder à un petit traitement pour générer ce fichier, et ensuite lancer Latex dessus. Sur plateforme Linux standard l'ensemble des outils nécessaires est normalement disponible, pour Windows, c'est faisable, mais un peu plus délicat.

D'abord, il faut expurger préalablement le fichier .csv des guillemets qui peuvent avoir été générés par le tableur (Il semble qu'OOo génère systématiquement ces guillemets autour des cellulles contenant du texte). On peut faire ça à la main via un "rechercher/remplacer" dans son éditeur favori, mais c'est plus élégant et surtout plus fiable de faire ça avec un outil dédié: sed

Depuis le shell, la commande
sed s/XXXX/YYYY/ input_file > output_file
va remplacer dans le fichier d'entrée input_file la première occurence de 'XXXX' par 'YYYY'. La sortie de programme se fait par défaut sur la "sortie standard" (la console), et il faut donc la rediriger vers le fichier souhaité (output_file) via le caractère de redirection '>'.

Pour remplacer toutes les occurences, on ajoute un 'g':
sed s/XXXX/YYYY/g input_file > output_file
Pour éliminer les occurences d'une chaine, il suffit de laisser YYYY vide.

En général, la première ligne du fichier contient le nom des colonnes, et il faut donc la supprimer. sed permet ceci avec la syntaxe:
sed 1d input_file > output_file
Pour regrouper les 2 commandes en une ligne, il faut ajouter l'option -e indiquant que la chaine qui suit est une commande:
sed -e 1d -e s/XXXX/YYYY/g input_file > output_file

Au final la commande est donc:
sed -e 1d -e s/\"//g ooo_export.csv > adresses.csv
(il faut "backslasher" le guillemet)

Un exemple de script bash est donné ci-dessous, qui enchaîne les 3 passes:
- preprocessing du fichier csv,
- génération du fichier d'adresses,
- appel de pdflatex pour la génération du pdf final.
Il faudra aussi adapter l'ordre des champs à ce que vous avez dans votre fichier d'origine.

#! /bin/bash

outfile=names.dat
sed -e 1d -e s/\"//g ooo_export.csv > adresses.csv

echo "" > $outfile

# var. spéciale: séparateur de champ
IFS=';'

while read LINE; do
 echo "$LINE";
 a=( $LINE )
 LNAME=${a[0]}
 FNAME=${a[1]}
 ADR=${a[2]}
 CODE=${a[3]}
 CITY=${a[4]}

 echo "$FNAME $LNAME"  >> $outfile
 echo "$ADR"  >> $outfile
 echo "$CODE $CITY"  >> $outfile
 echo "" >> $outfile

done < adresses.csv

pdflatex -interaction=batchmode my_labels.tex

Attention:
On a parfois dans les fichiers d'adresses des champs du style "Gérard & Jacqueline". Latex interprétant le '&' comme un caractère spécial, mieux vaut le supprimer de votre fichier source et remplacer par "Gérard et Jacqueline". Autre solution : "Gérard \& Jacqueline".

La suite ? Réaliser un export en ligne de commande du fichier d'adresses originel... Mais OOo ne semble pas disposer de cette fonctionnalité là, il faudrait passer par d'autres outils tiers.

Edit 10/2012: en fait, l'export en .ods  (Open Document Spreadsheet) depuis le fichier Excel de départ permet de disposer d'un fichier xml du contenu, dont on peut extraire le contenu. OpenDocument Fellowship propose des outils pour manipuler les fichiers Open Document, mais qui semblent à ce jour ne concerner que les fichiers odt (de type "texte", donc, et pas les feuilles de calcul ods).

L'autre approche consisterait à écrire un parser adéquat, qui récupère  les champs et les copie dans le fichier décrit ci-dessus.  pyxml semble l'outil le plus adapté, une piste à suivre...

Edit 02/2015: LibreOffice 4.2 (et OpenOffice ?) propose désormais un outil en ligne de commande qui permet d'exporter un fichier .ods en .csv, avec la syntaxe suivante:
libreoffice --headless --convert-to csv *.ods

lundi 2 avril 2012

GNU Make and the foreach() function

Make is a nice tool for building stuff. Software, of course, but not only, I use it also for building pdf files from Latex sources. However, its usage is not obvious at all, and many pitfalls lie in front of the newcomer. And the manual is not a great help when it comes to learning new features. Joyfully, there are bunches of tutorials out there.

However, some advanced tricks are rarely covered. Here is one trick that can be useful in some situations. Beware, it assumes you already know the basics about Make (GNU flavor), in case not, please go read some tutorial and come back in a while...

Lets say you have a folder containing some source files of type ".in", say foo.in and boo.in. These files need to be processed with some tool (call it "some_tool" for now) to build the output files. This (awesome) thing has a rather classical syntax:
some_tool in_file out_file [option]

Say you need to build these with two different configurations to produce two output files for each input files, say one of type "X" and one of type "Y". Here, from the two input files, you need to build:
foo_X.out
foo_Y.out
boo_X.out
boo_Y.out

How can Make help you so you only build the ones that are needed ? For the example, say that each build process takes several hours, so you really don't want to build them if it's not needed.

First create a variable holding all the input files:
infiles := $(wildcard *.in) 

Then, add a generic rule telling how to build a ".out" file:
%.out: %.in 
    @some_tool $< $@ 
Yes, remember these special variables: $< is the first dependency, and $@ is the target (%.out here), so that will generate the correct command-line. Oh, and for the different output flavors (say, by adding some option), just dupe the generic rule:
%_X.out: %.in
 @some_tool $< $@ X
%_Y.out: %.in
 @some_tool $< $@ Y
Finally, you will try to build the main (all) rule. Its dependencies are all the input files (thats easy, its $(infiles) ) and the output files, so that each of them gets produced only in case it is not present. Ah. Here comes some trouble. How do you create a variable holding all the expected targets ? Well, first, lets get the files names without extension. Easy:
filenames :=$(basename $(infiles))

Okay, and now, how do I automatically generate a string holding all the output files names ? This is a job for the foreach() function. Its description in the manual is (IMHO) mostly unreadable, so lets see this through a simpler example:
what = $(foreach a, ham cheese salad, john likes $(a) )
all:
    @echo "what=$(what)"
produces:
what= john likes ham  john likes cheese  john likes salad
Usually, this is used using some variables, and it would be written:
types= ham cheese salad
what = $(foreach a, $(types), john likes $(a) )

Mmmh, I'm sure you're starting to see the point. The 'types' will be our different output flavors, say, something like:
types= X Y
outfile := $(foreach a, $(types), boo_$(a).out )
     @echo "outfile=$(outfile)"
that will produce:
outfile= boo_X.out boo_Y.out
But this has to be done for each input files, and the following fails:
types= X Y
outfiles := $(foreach a, $(types), $(filenames)_$(a).out )
It gets expanded to:
boo foo_X.out boo foo_Y.out

So the solution is to use nested foreach() loops, just as you would do with some regular procedural programming language.
types:=X Y
outfiles := $(foreach a,$(filenames), \
     $(foreach b,$(types), \
         $(a)_$(b).out ) ) 
Then, the main rule can just get written as follows:
all: $(infiles) $(outfiles)
    @echo "That's all, folks !"

For further reading, check this series of articles on advanced topics on Make.

vendredi 17 février 2012

Changing the monitor on which the GNOME panels are placed

In my job, it is quite common these days to have a dual display setup on your laptop. Either your are at the office and have a larger LDC monitor, either your are teaching or doing some presentation and you have a beamer projector connected.

With the GNOME 2 desktop, the UI is based on 2 toolbars (called "panels") on top and bottom of the screen, that are very handy for launching common apps. With two screens, what happens is that these panels are usually placed on one screen, where you would rather have them on the other one.

The desktop provides a graphical way to move them around with the mouse, by holding the "alt" key down, and moving them around. But this is rather difficult in some lighting situations, when the projector image is bad and you hardly see your mouse pointer. Worse, you might end up with a vertical panel, and then, due to the lack of free space in the panel to click on, you're doomed to use either the graphical editor, either the command line tool.

The other solution is to edit this item directly in the gnome registry. Oh, sorry, I know, GNOME has no registry, only Windows has... Although it can look quite similar as viewed from the corresponding graphical editors, the implementation beneath is completely different. While Windows stores all its configuration information in two HUGE files, on Linux the information is spread over many many files, usually under /etc. It's only the way the information is displayed that makes it look like Windows registry (a tree of pairs key/values). And it is called GConf repository instead of registry.

For the desktops panels, the items to consider are stored under the keys /apps/panel/toplevels/ keys, one is called bottom_panel_screen0, the other top_panel_screen0 (at least on my computer, check on yours).

This can be done graphically with the gconf-editor tool, but when you have the audience waiting, it can be complicated finding and clicking on correct items... The best idea is to use the command-line tool provided, and to have somewhere two scripts, one for switching panels to monitor 0, one for switching to monitor 1.

So just copy/paste these lines in two files that you store whereyouwant, and add two symlinks on them on your desktop. Then, once you are in front of the audience and you have trouble with the panels, just reduce all the opened windows and double-click on one of the links. Changes should apply in less than a second.

file switch_panel_0.sh
#! /bin/bash
# switch top/bottom panels to screen 0
gconftool-2 --set "/apps/panel/toplevels/bottom_panel_screen0/monitor" --type int 0
gconftool-2 --set "/apps/panel/toplevels/top_panel_screen0/monitor" --type int 0

file switch_panel_1.sh:
#! /bin/bash
# switch top/bottom panels to screen 1
gconftool-2 --set "/apps/panel/toplevels/bottom_panel_screen0/monitor" --type int 1
gconftool-2 --set "/apps/panel/toplevels/top_panel_screen0/monitor" --type int 1

To create the symlinks (remember to adjust the path to where you put the files), just "cd" to your desktop folder ($HOME/Desktop usually) and type:
>ln -s $HOME/scripts/screen/switch_panel_0.sh panel_0
>ln -s $HOME/scripts/screen/switch_panel_1.sh panel_1

And don't forget to make your files executable! (chmod)

vendredi 2 décembre 2011

SLIFIS: fuzzy logic C++ library first release

I have just uploaded on sourceforge the SLIFIS library, a long-standing project on which I can say I have spend quite some time... It is a C++ library designed to be used inside scientific software when you need some fuzzy logic stuff (membership functions, fuzzy inference system, ...)

It all started in 2008-2009, when I was in that situation, and had trouble finding what would fit my needs. Since then, the software was more or less in a standby situation, but I fixed some stuff recently, and decided to make a public release. A lot of things still need to be worked on, I hope to clean up things a bit and will keep an eye on this.

Checkout the (reduced) doc here, and the download project page is here.

At present, Windows build is not up to date, as Linux is now my main dev platform, but it shouldn't be too hard to make it work.

As usual with open source, absolutely no warranty is given..., blah blah blah.

Feedback is welcome.

mardi 18 octobre 2011

C++: switch on any data type (part 2)

(See part 1 of this post here.)

In this second version, the idea is to have the same function called on triggering, but with a different argument value for each case.

The class will be very similar to first version. The difference is that here, instead of storing the function in the map, we store the argument value. i.e. instead of having
std::map<KEY, FUNC>, we change to  std::map<KEY, ARG>.

Here is the full class. Don't forget to add the needed headers. And as usual with templated classes, this takes place in a header file.


template < typename KEY, typename FUNC, typename ARG > class SWITCH2
{
   public:
      SWITCH2()
      {
          myFunc = 0;
      }

      void Process( const KEY& key )
      {
         assert( myFunc ); // needed !
         typename std::map<KEY,ARG>::const_iterator it = myMap.find( key );
         if( it != myMap.end() )    // If key exists, call function
            myFunc( it->second );  // with correct argument
         else                       // else, call function with
            myFunc( DefArg );      // default argument
      }

      void AddArgument( const KEY key, ARG value )
      {
         myMap[ key ] = value;
      }
      void SetFunction( FUNC f )
      {
         myFunc = f;
      }
      void SetDefault( ARG def )
      {
         DefArg = def;
      }

   private:
      std::map< KEY, ARG > myMap;
      FUNC myFunc; // the function
      ARG  DefArg; // default value, the type needs
                   //   a default constructor
};

I removed here the uniqueness test, as it seemed less relevant in this situation. But you can write it back if needed.
Also note the 'assert()' checking, to make sure we don't call a function that hasn't been defined. You can easily replace this by some custom error handling function instead (exception, or whatever.)


And again, remember that this trick can only be used in specific cases where either you have to call different functions with the same argument, or the same function with different argument values.

Other solutions to this problem can be found (besides switching to another programming language, I mean...), you could also use function pointers, although this is less natural in C++.

mardi 14 juin 2011

C++: switch on any data type (part 1)

C++ is claimed to be a programming langage that allows Object Oriented programming. But for legacy reasons, it does not always allow "real" objet paradigm in certain situations. Yes, I'm talking about the "switch" statement.
As you might know, you can only switch on data types that can be downcasted to an 'int' type.

This comes of course from its proximity with C (that is sometimes considered as some upgrated assembler), and C++ programmers are now used to it. But from a strict Object Oriented paradigm, I believe this shouldn't be true.

But when would such a feature be needed anyway ? I'll show you an example.

Say you need to read data from a file that can be in two different formats. Say xml, csv, whatever. This will typically be done with two different functions. So the first step is identifying file type (by its extension, lets keep it simple), then call the correct function. So you have the following code:

if( ext == "xml" )
   return ReadFile_xml( ...
else
   if( ext == "csv" )
      return ReadFile_csv( ...
   else
      cerr << "Unrecognised file type !\n"; // or whatever other error handling

Okay, so, yes, this does the job. But what if you need to add two other file types and their associated functions ? The code before will start to look really really ugly and will become error prone. It would be much nicer if one could have this instead:
(for clarity, we have removed the handling of the returned value)

switch( ext )
{
   case "csv" : ReadFile_csv( ... ); break;
   case "xml" : ReadFile_xml( ... ); break;
   case "..." : ...
   ...
   default:
      cerr << "Unrecognized file type !\n";
}
return false;

Unfortunatly, as explained before, you can't do this in C++. No, Nada, Niet.

The example here is for strings, but can be transposed to a lot of similar situations. Say, how about some keyboard shortcut combination that you would want to switch on, or whatever.

What we would like is to have a "generic" solution that can be used for any data type. We present here two class-based solutions, one where you can call different functions for different "cases", the second where it is always the same function called, but with different argument values. They rely on template programming, with a provided class that holds all the necessary logic and can be reused elsewhere, just put the class in some header file and use it anywhere.

These solutions both have limitations at present. For the first flavour, the number of arguments is fixed in the templated class. You can of course edit it, but it is not fully generic at present.

Let's see first how the user code will look using the first solution. We show here an example where the argument to be passed to the function has the type string, but the point here is that this solution is independent of both types used (selector, and argument), as long as they meet certain expectations, see below.

// first, instanciate the class SWITCH with needed types
//                key type        function sig      arg type
    SWITCH mySwitch< string,   void(*)(const string&), string >;
// then, configure the "switch" object
    mySwitch.Add( "csv", ReadFile_csv );
    mySwitch.Add( "xml", ReadFile_xml );
    mySwitch.AddDefault( ReadFile_unknown );

// lets try some keyboard test
    string in;
    do
    {
        cout << "in: ";
        cin >> in;
        mySwitch.Process( in, filename ); // do the switch !
    }
    while( in != "0" );

   ...
}
// this function needs to have the following signature
void ReadFile_csv( const string& filename )
{
 ...
}

Now, lets see the class. It is based on an stl map. It is templated by three types, the key type (the "selector"), the function signature, and the functions argument type:

template < typename KEY, typename FUNC, typename ARG > class SWITCH
{
   public:
      SWITCH()
      {
         Def = 0; // no default function at startup
      }

      void Process( const KEY& key, ARG arg )
      {
         typename std::map< KEY, FUNC >::const_iterator it = my_map.find( key );
         if( it != my_map.end() )  // If key exists, call
             it->second( arg );    // associated function
         else               // else, call
            if( Def )       // default function, if there is one.
               Def( arg );  // Else, do nothing
      }

      void Add( const KEY& key, FUNC my_func )
      {
#ifdef CHECK_FOR_UNIQUENESS
         typename std::map< KEY, FUNC >::const_iterator it = my_map.find( key );
         if( it != my_map.end() )
         {
            cerr << "Already defined !\n";
            throw "Already defined !\n";
         }
#endif
         my_map[ key ] = my_func;
      }

      void AddDefault( FUNC f )
      {
          Def = f;
      }

   private:
      std::map< KEY, FUNC > my_map;
      FUNC Def; // default function
};

Some comments:
  • Please note the CHECK_FOR_UNIQUENESS test. If defined, then execution stops if class user attemps to store twice the same key. You can easily write your own error handler.
  • The KEY and the ARG type need to be copyable. This is a limitation. For instance, you could not use a file stream (std::ifstream for example) as these are not copyable. If you use a non-trivial class, make sure its assignement operator is defined.
  • As you may have noticed, the functions argument can be passed using its reference.

Part 2 is coming soon, in the meanwhile, you can check the following references on the same -or similar- subject: