A guest post by Tanya Chhabra, a contributor to the Accord Project Summer of Code 2022.
- Project overview
- Community bonding period
- The coding journey –
- Understanding the models
- Writing up the transformation rules
- Improving user experience – the challenges
- What lies ahead?
- Learnings and experience
Project Overview
The Accord Project uses its own transformation framework to convert the Cicero Template Archive (.cta) for smart contracts into Markdown (.md) for lightweight rich-text markup, PDF, JSON, etc. This framework is a compilation of around 10 packages that are interlinked and work together to make the transformations possible. However, none of these packages were compatible to handle the conversion of ‘tables’ from and to different file formats.
The primary goal of my project was to include tables in each of these packages and enable roundtrip among all of them.
Community Bonding Period
I used the community bonding period to get familiar with various transform packages and their working. I also decided to split the upcoming 6 weeks into two phases and divided my tasks accordingly. Some discussions were made regarding the flow of the project and it was decided that it will be most beneficial to start the journey from the ‘CommonMark transform’ as it was the root package and all the other transform were built keeping markdown-common as the base. Furthermore, I set up a demo server of my own so as to test the working of the code written in the future.
The Summer of Code
The official coding period began on 14th July, 2022 and I had about 6 weeks to incorporate tables into 5 transform packages, write a unit tests for each of them and finally link them up with the markdown-UI editor so that the users could easily work with tables.
Understanding the models
The first step of this whole process was to analyse and understand various markdown models and incorporate tables into them because all the transformation rules were written keeping the models into mind. They have been built to provide a schema for each node in the package.
I began by working on the commonmark model (version 0.2.0) and experimenting with different nodes like lists, images, etc. I looked into what were the necessary attributes for a simple markdown table, compared it with the HTML structure of a table and after some trial and error, a common mark model for tables was ready.
After some scrutiny, it was realised that there was no pressing need to update the ciceromark or templatemark models as these models were already importing all the nodes described in commonmark model.
However, these models had to be edited to use the new version of commonmark.
The transformation rules
There were 4 packages in total that had to be updated –
- markdown-common ( markdown <-> CommonMark DOM )
- markdown-html ( CiceroMark DOM <-> HTML )
- markdown-slate ( CiceroMark DOM <-> Slate DOM )
- markdown-pdf ( CiceroMark DOM <-> PDF )
In each of these packages, rules for all the 6 table nodes had to be specified and tested for roundtrip. I began by experimenting and analysing the working of a particular package through my demo server and the markus CLI. After that, 2 sets of rules were written for all the 4 transformations.
The conversion process for markdown-slate:
This package specifies 3 sets of rules for transformations of slate-DOM from and to the commonmark, the ciceromark and the templatemark respectively. The only thing that had to be done was to include table rules in the commonmark <-> slate DOM because, as explained, the ciceromark and templatemark are just extensions of the commonmark DOM.
As can be seen here, the CiceroMarkToSlateVisitor ‘extends’ the ToSlateVisitor which, in turn, is based on commonmark rules.
The conversion process for markdown-pdf:
This was an interesting transformation as the commonmark table had to be converted into the specific format described by ‘pdfmake’ as can be seen here. This was again done by studying the model and both the formats and the result is as shown below:
These rules were tested out by trying to convert a complex commonmark table along with distinct inline elements into a PDF.
It is important to note here that the basic conversions namely, markdown <-> CommonMark DOM and CiceroMark DOM <-> HTML were completed beforehand in a similar manner so as to easily complete the above mentioned transformations.
Furthermore, after all the transformations were complete, unit tests were written for each of them to ensure proper working and eliminate any potential errors.
Improving user experience
The last stage of the project was to enable the users to effectively use the added feature of tables in the markdown-UI editor. It should be noted here that the UI editor had already been customised to create and add a table support. Thus, combining that feature with the transformations mentioned, allowed the user to easily copy or paste tables.
However, some challenges were faced while combining these two parts:
First of all, a markdown string does not support a table without a header and treats it just like a paragraph. However, tables without headers could be pasted into the editor by a user. Therefore, I had to make sure that such a table was supported in the editor even though markdown did not support it.
Moreover, the table schema in the slate editor had to be updated to incorporate the commonmark model so as to smoothen the roundtrip conversion and improve the UI.
Even after all these changes, the layout of the table in the editor was a bit bizarre even though the copying and pasting was working. The main issue was that on pasting a table into the editor, the ‘table’ node itself was lost. This issue was solved by editing the ‘withHtml’ plugin to ‘insert the table node’ instead of ‘table fragment’.
Thus, the copy/paste functionality was proof that transformations were working for converting a table from Markdown to HTML to Slate DOM and vice versa.
After this, to demonstrate the working of PDF transformations, Diana installed ‘export buttons’ in the editor that allowed a user to directly download the editor content in the form of markdown, HTML or PDF.
What lies ahead?
Even though all of the required transformations now support tables and the UI editor has been updated, there are still a few features that can be incorporated to further this project such as:
- Tables in DOCx transformations.
- The ability to edit table variables in Templatemark.
- Improving the ‘cut’ functionality in the editor.
Result
APSoC Experience and Learnings
APSoC was an amazing experience for me and I learned a lot during these six weeks. Every member of the community was extremely supportive and always ready to help. I would specially like to thank both of my mentors, Dan and Diana who were by my side throughout the process and helped me with everything. Through this project, I was able to improve my programming skills, my problem solving skills as well as my soft skills. I learned how to manage a given project within a certain timeframe and how to present the work done by me in a creative manner.
I will try my best to remain an active member of this community and contribute as much as I can.