- Status Closed
- Percent Complete
- Task Type Bug Report
- Category QCAD (main)
-
Assigned To
Andrew - Operating System All
- Severity Low
- Priority Defer
- Reported Version 3 TP1
- Due in Version Post 3.0
-
Due Date
Undecided
- Votes
- Private
Opened by Peter - 19.06.2011
Last edited by Andrew - 16.12.2013
FS#117 - Use APPID when handling extended entity data to prevent dataloss
Rationale:
Let QCAD handle extended entity data, that is used in the industry to instruct postprocessing applications, e.g. the tooling of a mill or robot.
When I say group code or specs below, I refer to this document http://www.autodesk.com/techpubs/autocad/acad2000/dxf/extended_data_dxf_ab.htm
Current:
In the property editor I can add key value pairs to drawing entities. As in this script:
entity.setProperty(new RPropertyTypeId("MY_KEY"), "MY_VAL"); entity.setProperty(new RPropertyTypeId("MY_REAL"), 123);
In DXF this results in a group code “QCAD” followed by alternating keys and values as strings and typed positions respectively:
[…] 0 LINE […] 1001 QCAD 1000 MY_KEY 1000 MY_VAL 1000 MY_REAL 1040 123.0 […]
Expected:
From reading the specs, this looks perfectly valid. I’d say though, that QCAD is not right to expect any extended data to follow this pattern. The extended entity data, that I want to recreate eg, looks like this:
[…] 0 LINE […] 1001 MY_DATA 1000 MY_FOO: some text 1000 MY_BAR: 1.250000 […]
That is: a group code, followed by some string values. Notice that the (limited) typing facilities mentioned in the spec are not used, but the application relies on its own peculiar parsing. The same group code appears in the head of the document as an APPID:
[…] 0 APPID 2 MY_DATA 70 0 0 […]
Proposed:
I suggest, that QCAD uses APPIDs to group extended data. Otherwise data loss may occur. In order to not have to maintain a table of known APPIDs, that tells how to parse their extended data, I further suggest that QCAD not expect extended data to follow a “key value” pattern but an “APPID entries” pattern. Here too data would be lost, if there was an odd number of entries in the extended table and QCAD insisted on its own approach.
Within the specs, deep trees could be constructed with the use of the control string (code 1002, { and }), and the propertyEditor would become a nightmare;) So I propose, that QCAD might parse only extended data with its own APPID in key value pairs, and data with other APPIDs as just an ordered list of entries. (In ECMAscript an array will have to be used, as object properties do not keep sequence.)
16.12.2013 13:20
Reason for closing: Fixed
Additional comments about closing:
QCAD 3.4 uses the key:value notation for
XData
I would really like to see this in the final release, if not in the next TP. I suspect lots of apps use extended parameters in a way that is incompatible with QCADs interpretation. So I made a mockup of how the property editor might look like. Currently (TP1) it reads "Custom" twice:
Custom properties are grouped by their APPID. QCAD ones are key value pairs, MyMill ones are just positional parameters - So sequence matters: in TP1 records are sorted alphabetically, that will not work with non-named entities. The "add" dialog should sport an additional drop down menu to select the group from known APPIDs.
In script this can be exposed as a method in RDocument to list (maybe even add/delete) APPIDs. The setPropoerty method of drawing entities might be overloaded so it takes two or three parameters. The APPID, the (optional (for QCAD style extended attributes)) key, a (single, typed) value.
Due to the complexity involved with this and the relatively small number of users requiring this feature, we will likely not support XDATA (custom properties) for the first release of QCAD 3.
In Beta handling of extended entity data was removed. If such data is in a file that QCAD handles, it will be lost when saving the file, and QCAD will remain silent on that.
The aCad way of specifying extended entity data is indeed peculiar, and I have been thinking about how to best represent it in a graphical interface. I think, it is a tree with the root node hidden, where the appIDs make up branches, and the typed data makes leaves of the kinds in the spec. Depth is limited to that: the root can only grow branches, the branches can only grow leaves. Leaves preserve order amongst them. This is the simplest model I can come up with.
I just keep adding to this, though I do not expect a quick resolve, especially, as I am beginning to see the complexity of xdata. I wrote down the spirit of the example from the specification, plus the TP of QCAD3 use of xdata as JSON:
The sample shows mostly static data, but there are some "codes", whose values will be affected by changes to the entity, to which the xdata is attached (1011, 1021, 1031, 1012, 1022, 1032, 1013, 1023, 1033, 1041, 1042). The closing brace at the end is not missing, AutoCAD will insert it. Code 1005, if I understand correctly, references other entities in a drawing. Although it is programmatically easy to tack such an object on to an entity from a QCAD script, a fully conforming implementation would have to translate some codes in certain behaviour, which obviously is quite a burden.
The TP of QCAD3 use of xdata indeed conforms to the specification. That these are alternating key value pairs is of no interest for applications other than QCAD or applications built on top of QCAD. Of course this assumption will only hold for the QCAD group, not for the AME_SOL group, just as the sequence of entries in the AME_SOL table only will be useful to applications that know how it is to be interpreted. But both QCAD and AME_SOL can rely upon the specification, that conforming applications will not make a mess of its peculiar xdata (at least the one in static codes).
Thanks. I appreciate your research into this!
Andrew, you are too kind. I am always a little afraid of leaning too far out of the window. Of course I do this on the agenda of my use case, namely instructing a robot programming application. For completeness what xdata it expects including the above - again as JSON, seen as just another serialization of what the DXF spec prescribes:
It was undoubtedly nice, if qcad 3 final prompted the user with a notice, that xdata will be lost in files saved from qcad, if it encounters a code 1000 on any entity in a file it loads.
Later then, when custom entity data gets real, you might want to limit this message to the more involved types. Here a whitelist of types that are save, when not touched: 1000, 1002, 1004, 1010, 1020, 1030, 1040, 1070, 1071. All others require qcad to follow certain rules to keep them up to date, 1005 might require a hook into entity deletion, 1003 into layer renaming.
The scripting API should stay close to the serialization. Particularly, a qcad script developer should be able to create her own appID and assign xdata of that appID to entities, which may look like any of the above, except maybe the 1005 codes. Maintaining the model, the way an apps xdata is structured, should be a separate task. I suppose for example, that 1010, 1020, 1030 always have to immediately follow one another. That should be left to the app developer. qcad may offer convenience functions to scripters, that implement its preferred way of organizing xdata below its own QCAD appID.
So far, the only difference between this and what TP1 offered, is the addition of quasi low level xdata handling and custom appID creation - As a bonus I imagine the possibility of binding the choice of GUI used for editing xdata to the appID. Scripters would extend the prototype of a basic editor into one, that displays the appropriate controls and keeps the model intact. The property editor would not display "Custom", but instead "QCAD", and any registered editor. The most basic editor I imagine as just a black hole.
You pulled support for xdata from the BETA releases because it wrecked havock. Maybe a separation like that could help, especially if the core constrained itself to the low level stuff.
Peter, spending the evenig building castles in the sky
QCAD 3.4.0 has been released today. This is the first QCAD release with custom properties enabled by default. Custom properties will be used for QCAD/CAM in the future to 'tag' entities with various machine processing options (tool radius correction inside or outside, contour start, pocket processing, point is a drill hole, etc).
3rd party custom properties are supported for types 'string', 'int' and 'double' at this point.
Andrew, only now I have found the time to look at this: What I have found to work is adding custom properties that have the "QCAD" AppId. That is, the "title" from the function signature: entity.setCustomProperty(title, key, value).
When I use another AppId, I see it in the table of AppIds in the head of the resulting file, but the custom properties (that make use of it) on entities set in the creation of the drawing do not get serialized to DXF.
Also I see, that the properties are alphabetically sorted in the file. I believe though, that in the specification, custom properties are like positional parameters, so order their sequence should be kept in order.
Maybe this is academic, possibly it would be good to close this convoluted report and open a fresh request issue?
Pater: custom properties with other AppIDs would have to be added in the following format to be stored in DXF / DWG:
Title (AppID): "MyApp" (or anything else)
Key: "0001_1000" where 0001 is the sequence order and 1000 is the DXF code to be used to store the property in DXF / DWG
Value: appropriate value (only string, int and double are supported at this point)
Note that this is how QCAD stores 3rd party custom properties internally. This might be subject to change in the future. DXF / DWG files would not be affected by such a change in the future.
I see, when I call the function like this:
then the resulting DXF indeed has the correct information.
I understand that qCAD internally parses the key and that this is a little fragile and this "undocumented feature" therefore subject to change.
Thank you still!
Yes, perhaps it's a good idea to wrap your calls to setCustomProperty into your own function, so you only have to refactor one function in case the QCAD API changes in the future:
Danke Andrew, below my take on the helper:
From a cursory test, the handling of extended data seems fair to me: while QCAD stores its own in key:value pairs, already existing xdata is copied as is and in sequence from the input to the /teigha/ output. So there is no data lost by opening and saving a drawing in QCAD-PRO, at least if the codes are not of the dynamic types – which might be unreasonable to expect, and QCAD prompts the user then anyways I think, in case something is lost. Good work!