QGIS Logo

The Problem with QGIS and csv Files

Last month I was at the Maptime in Southampton (UK), helping QGIS new users with how to join tables and map EU referendum maps, when I came across an issue with something on QGIS I hadn’t spotted in the last *ahem* years of using it.

When you drag and drop txt, csv, or other delimited files into QGIS, the fields automatically get converted to text format. No, I’m not making it up, and it caused a lot of embarrassment when I was giving my demonstration.

draganddrop fields

By dragging and dropping the csv file, you can see that the field type is solely “String”

 

This isn’t written to complain about QGIS but to notify others who are wondering why their joins aren’t working or why their interpolation can’t pick up the value field. You QGIS guys are going to say, “Why haven’t I raised this as an issue?” Well, firstly read Nyall Dawsons blog post on QGIS issues. Secondly, I tried to. It turns out that trying to get access to submit issues has changed, and even though I’ve asked for help to get access I’ve been waiting one month for response to request.

So, why does it happen?

If you add the file through the “add delimited file” button, none of this is an issue. This is due to how the software is written. When the file is “dragged & dropped”, the software relies on OGR to add it as a comprehensible layer, and this just renders all the fields as text (at present, August 2016).

Add layer fields e2e

By adding the csv file using the add layer method, you can see the fields are brought in correctly

Why is it an issue?

If you are joining tables and aren’t aware of the issue, you drag and drop a table with a list of numerical values in and then can’t join it to a spatial data with values in, as you can’t join text to numbers. This could also cause issues with interpolation (reading of a value field) and generation of points that need classification based on numbers.

Getting it fixed

This is where things get a little tricky as I don’t think it is entirely a QGIS issue and more related to the code that QGIS uses to parse the information, so until OGR update their code, it might be a bit of a wait.

 

Dragons8mycat

The Problem with QGIS and csv Files” Comments

  1. […] time you have a .csv or .txt file to open in QGIS; this is what will happen to you. Read through Duggans article on his experience with this…just to give you a heads up Duggan was demonstrating to QGIS new […]

  2. did you solved problems related with OSGeo id to notify issues?
    Do you still need a mantra word?

    • Dragons8mycat Dragons8mycat

      Hi Luigi,
      No, the OSGeo issues are still not resolved. I have a mantra which I tried but it is telling me that I have a duplicate email.
      I think it has tripped over itself at some point, I’ve asked OSGeo to look at it but have had no response. A shame really as I have just got OSTN15 working in Proj.4 and was going to put it into QGIS.

Leave a Reply

Your email address will not be published. Required fields are marked *