QCAD Bugtracker

  • Status Closed
  • Percent Complete
    100%
  • Task Type Bug Report
  • Category QCAD (main)
  • Assigned To
    Andrew
  • Operating System All
  • Severity Low
  • Priority Defer
  • Reported Version 3 TP1
  • Due in Version Post 3.0
  • Due Date Undecided
  • Votes
  • Private
Attached to Project: QCAD Bugtracker
Opened by Peter - 19.06.2011
Last edited by Andrew - 16.12.2013

FS#117 - Use APPID when handling extended entity data to prevent dataloss

Rationale:

Let QCAD handle extended entity data, that is used in the industry to instruct postprocessing applications, e.g. the tooling of a mill or robot.

When I say group code or specs below, I refer to this document http://www.autodesk.com/techpubs/autocad/acad2000/dxf/extended_data_dxf_ab.htm

Current:

In the property editor I can add key value pairs to drawing entities. As in this script:

entity.setProperty(new RPropertyTypeId("MY_KEY"), "MY_VAL");
entity.setProperty(new RPropertyTypeId("MY_REAL"), 123);

In DXF this results in a group code “QCAD” followed by alternating keys and values as strings and typed positions respectively:

[…]
  0
LINE
[…]
1001
QCAD
1000
MY_KEY
1000
MY_VAL
1000
MY_REAL
1040
123.0
[…]

Expected:

From reading the specs, this looks perfectly valid. I’d say though, that QCAD is not right to expect any extended data to follow this pattern. The extended entity data, that I want to recreate eg, looks like this:

[…]
  0
LINE
[…]
1001
MY_DATA
1000
MY_FOO: some text
1000
MY_BAR: 1.250000
[…]

That is: a group code, followed by some string values. Notice that the (limited) typing facilities mentioned in the spec are not used, but the application relies on its own peculiar parsing. The same group code appears in the head of the document as an APPID:

[…]
  0
APPID
  2
MY_DATA
 70
     0
  0
[…]

Proposed:

I suggest, that QCAD uses APPIDs to group extended data. Otherwise data loss may occur. In order to not have to maintain a table of known APPIDs, that tells how to parse their extended data, I further suggest that QCAD not expect extended data to follow a “key value” pattern but an “APPID entries” pattern. Here too data would be lost, if there was an odd number of entries in the extended table and QCAD insisted on its own approach.

Within the specs, deep trees could be constructed with the use of the control string (code 1002, { and }), and the propertyEditor would become a nightmare;) So I propose, that QCAD might parse only extended data with its own APPID in key value pairs, and data with other APPIDs as just an ordered list of entries. (In ECMAscript an array will have to be used, as object properties do not keep sequence.)

Closed by  Andrew
16.12.2013 13:20
Reason for closing:  Fixed
Additional comments about closing:  

QCAD 3.4 uses the key:value notation for XData

Peter commented on 20.06.2011 09:49

I would really like to see this in the final release, if not in the next TP. I suspect lots of apps use extended parameters in a way that is incompatible with QCADs interpretation. So I made a mockup of how the property editor might look like. Currently (TP1) it reads "Custom" twice:

Custom properties are grouped by their APPID. QCAD ones are key value pairs, MyMill ones are just positional parameters - So sequence matters: in TP1 records are sorted alphabetically, that will not work with non-named entities. The "add" dialog should sport an additional drop down menu to select the group from known APPIDs.

In script this can be exposed as a method in RDocument to list (maybe even add/delete) APPIDs. The setPropoerty method of drawing entities might be overloaded so it takes two or three parameters. The APPID, the (optional (for QCAD style extended attributes)) key, a (single, typed) value.

Admin
Andrew commented on 14.07.2011 14:05

Due to the complexity involved with this and the relatively small number of users requiring this feature, we will likely not support XDATA (custom properties) for the first release of QCAD 3.

Peter commented on 06.09.2011 11:38

In Beta handling of extended entity data was removed. If such data is in a file that QCAD handles, it will be lost when saving the file, and QCAD will remain silent on that.

Peter commented on 13.12.2011 19:04

The aCad way of specifying extended entity data is indeed peculiar, and I have been thinking about how to best represent it in a graphical interface. I think, it is a tree with the root node hidden, where the appIDs make up branches, and the typed data makes leaves of the kinds in the spec. Depth is limited to that: the root can only grow branches, the branches can only grow leaves. Leaves preserve order amongst them. This is the simplest model I can come up with.

Peter commented on 14.12.2011 19:38

I just keep adding to this, though I do not expect a quick resolve, especially, as I am beginning to see the complexity of xdata. I wrote down the spirit of the example from the specification, plus the TP of QCAD3 use of xdata as JSON:

{
  "AME_SOL": [
    { code: "1002", value: "{"         }, // Control String
    { code: "1070", value: 0"          }, // Integer
    { code: "1071", value: 1.95059E+06 }, // Long
    { code: "1070", value: 519         }, // Integer
    { code: "1010", value: 2.54717     }, // X
    { code: "1020", value: 2.122642    }, // Y
    { code: "1030", value: 2.049201    }, // Z
    { code: "1005", value: "ECD"       },
    { code: "1005", value: "EE9"       },
    { code: "1005", value: 0           },
    { code: "1040", value: 0.0         }, // Real
    { code: "1040", value: 1.0         }, // Real
    { code: "1000", value: MILD_STEEL  }  // String
  ],
  "QCAD": [
    { code: "1000", value: "MY_KEY"    },
    { code: "1000", value: "MY_VALUE"  },
    { code: "1000", value: "MY_INT"    },
    { code: "1070", value: 0"          },
    { code: "1000", value: "MY_REAL"   },
    { code: "1040", value: 0.0         }
  ]
}

The sample shows mostly static data, but there are some "codes", whose values will be affected by changes to the entity, to which the xdata is attached (1011, 1021, 1031, 1012, 1022, 1032, 1013, 1023, 1033, 1041, 1042). The closing brace at the end is not missing, AutoCAD will insert it. Code 1005, if I understand correctly, references other entities in a drawing. Although it is programmatically easy to tack such an object on to an entity from a QCAD script, a fully conforming implementation would have to translate some codes in certain behaviour, which obviously is quite a burden.

The TP of QCAD3 use of xdata indeed conforms to the specification. That these are alternating key value pairs is of no interest for applications other than QCAD or applications built on top of QCAD. Of course this assumption will only hold for the QCAD group, not for the AME_SOL group, just as the sequence of entries in the AME_SOL table only will be useful to applications that know how it is to be interpreted. But both QCAD and AME_SOL can rely upon the specification, that conforming applications will not make a mess of its peculiar xdata (at least the one in static codes).

Admin
Andrew commented on 14.12.2011 19:47

Thanks. I appreciate your research into this!

Peter commented on 14.12.2011 22:29

Andrew, you are too kind. I am always a little afraid of leaning too far out of the window. Of course I do this on the agenda of my use case, namely instructing a robot programming application. For completeness what xdata it expects including the above - again as JSON, seen as just another serialization of what the DXF spec prescribes:

{
  "AME_SOL": [
    { code: "1002", value: "{"         }, // Control String
    { code: "1070", value: 0"          }, // Integer
    { code: "1071", value: 1.95059E+06 }, // Long
    { code: "1070", value: 519         }, // Integer
    { code: "1010", value: 2.54717     }, // X
    { code: "1020", value: 2.122642    }, // Y
    { code: "1030", value: 2.049201    }, // Z
    { code: "1005", value: "ECD"       },
    { code: "1005", value: "EE9"       },
    { code: "1005", value: 0           },
    { code: "1040", value: 0.0         }, // Real
    { code: "1040", value: 1.0         }, // Real
    { code: "1000", value: MILD_STEEL  }  // String
  ],
  "QCAD": [
    { code: "1000", value: "MY_KEY"    },
    { code: "1000", value: "MY_VALUE"  },
    { code: "1000", value: "MY_INT"    },
    { code: "1070", value: 0           },
    { code: "1000", value: "MY_REAL"   },
    { code: "1040", value: 0.0         }
  ],
  "MY_MILL" : {
    { code: "1000", value: "MY_FOO: some text" },
    { code: "1000", value: "MY_BAR: 1.250000"  }  
  }
}

It was undoubtedly nice, if qcad 3 final prompted the user with a notice, that xdata will be lost in files saved from qcad, if it encounters a code 1000 on any entity in a file it loads.

Later then, when custom entity data gets real, you might want to limit this message to the more involved types. Here a whitelist of types that are save, when not touched: 1000, 1002, 1004, 1010, 1020, 1030, 1040, 1070, 1071. All others require qcad to follow certain rules to keep them up to date, 1005 might require a hook into entity deletion, 1003 into layer renaming.

The scripting API should stay close to the serialization. Particularly, a qcad script developer should be able to create her own appID and assign xdata of that appID to entities, which may look like any of the above, except maybe the 1005 codes. Maintaining the model, the way an apps xdata is structured, should be a separate task. I suppose for example, that 1010, 1020, 1030 always have to immediately follow one another. That should be left to the app developer. qcad may offer convenience functions to scripters, that implement its preferred way of organizing xdata below its own QCAD appID.

So far, the only difference between this and what TP1 offered, is the addition of quasi low level xdata handling and custom appID creation - As a bonus I imagine the possibility of binding the choice of GUI used for editing xdata to the appID. Scripters would extend the prototype of a basic editor into one, that displays the appropriate controls and keeps the model intact. The property editor would not display "Custom", but instead "QCAD", and any registered editor. The most basic editor I imagine as just a black hole.

You pulled support for xdata from the BETA releases because it wrecked havock. Maybe a separation like that could help, especially if the core constrained itself to the low level stuff.

Peter, spending the evenig building castles in the sky

Admin
Andrew commented on 23.10.2013 10:00

QCAD 3.4.0 has been released today. This is the first QCAD release with custom properties enabled by default. Custom properties will be used for QCAD/CAM in the future to 'tag' entities with various machine processing options (tool radius correction inside or outside, contour start, pocket processing, point is a drill hole, etc).

3rd party custom properties are supported for types 'string', 'int' and 'double' at this point.

Peter commented on 25.10.2013 12:45

Andrew, only now I have found the time to look at this: What I have found to work is adding custom properties that have the "QCAD" AppId. That is, the "title" from the function signature: entity.setCustomProperty(title, key, value).

When I use another AppId, I see it in the table of AppIds in the head of the resulting file, but the custom properties (that make use of it) on entities set in the creation of the drawing do not get serialized to DXF.

Also I see, that the properties are alphabetically sorted in the file. I believe though, that in the specification, custom properties are like positional parameters, so order their sequence should be kept in order.

Maybe this is academic, possibly it would be good to close this convoluted report and open a fresh request issue?

Admin
Andrew commented on 25.10.2013 13:21

Pater: custom properties with other AppIDs would have to be added in the following format to be stored in DXF / DWG:

Title (AppID): "MyApp" (or anything else)
Key: "0001_1000" where 0001 is the sequence order and 1000 is the DXF code to be used to store the property in DXF / DWG
Value: appropriate value (only string, int and double are supported at this point)

Note that this is how QCAD stores 3rd party custom properties internally. This might be subject to change in the future. DXF / DWG files would not be affected by such a change in the future.

Peter commented on 25.10.2013 14:00

I see, when I call the function like this:

entity.setCustomProperty("MyMill", "0001_1070", 123) // int
entity.setCustomProperty("MyMill", "0002_1000", "X: 123") // str

then the resulting DXF indeed has the correct information.

I understand that qCAD internally parses the key and that this is a little fragile and this "undocumented feature" therefore subject to change.

Thank you still!

Admin
Andrew commented on 25.10.2013 14:30
I understand that qCAD internally parses the key and that this is a little fragile and this "undocumented feature" therefore subject to change.

Yes, perhaps it's a good idea to wrap your calls to setCustomProperty into your own function, so you only have to refactor one function in case the QCAD API changes in the future:

"%1_%2"'0'
Peter commented on 02.11.2013 13:00

Danke Andrew, below my take on the helper:

/** Ein gegebenes Objekt mit erweiterten Daten versehen
 * @param {Object} entity das Zeichenobjekt
 * @param {String} group Gruppencode des Progamms
 * @param {Array} preset die Daten
 * @see QCAD issues FS#117, FS#425
 */"0000""_1000"
Peter commented on 17.12.2013 12:06

From a cursory test, the handling of extended data seems fair to me: while QCAD stores its own in key:value pairs, already existing xdata is copied as is and in sequence from the input to the /teigha/ output. So there is no data lost by opening and saving a drawing in QCAD-PRO, at least if the codes are not of the dynamic types – which might be unreasonable to expect, and QCAD prompts the user then anyways I think, in case something is lost. Good work!

Loading...

Available keyboard shortcuts

Tasklist

Task Details

Task Editing