Custom IntelliSense with Monaco Editor

Development | Ivana Simic

Custom IntelliSense with Monaco Editor

Tuesday, Apr 11, 2017 • 14 min read

Extend the XML IntelliSense feature of Monaco Editor with custom completion item provider.

Monaco Editor comes with great features we can use to improve the experience of coding. In this post, we’ll use one of its features to extend IntelliSense for XML.

Since Monaco editor is generated from Visual Studio Code’s sources, it has a lot of VS Code feel to it. Like VS Code, Monaco comes with built-in syntax colorization for a few languages, as well as IntelliSense and validation for TypeScript, JavaScript, CSS, LESS, SCSS, JSON, and HTML. Still, you can override those built-in features with your own - or create new ones, for example, create custom code suggestions for the language of your choice.

Creating IntelliSense for XML

Since XML is one of the languages that don’t have built-in IntelliSense, let’s create it now. There are a lot of ways to improve or extend IntelliSense. For example, you can create a hover provider to control what happens when you hover over a part of code, a definition provider to control the go to definition behavior, an implementation provider to control the go to implementation behavior, and many others. But, let’s keep it simple, and create a custom completion item provider. The idea here is to have an XSD schema with our definitions and to use it to provide completions while writing XML code.

Completion item provider - intro

Completion item provider defines the rules for the suggestions our editor will offer in different parts of the code. It defines the contract between extensions and the IntelliSense and can consist of triggerCharacters property and two methods - provideCompletionItems and resolveCompletionItem. We will explain all of those along with the code, to make it easier to reason about them.

The end goal

Before starting our project and looking into any code, it’s good to know what do we want to achieve. We know that an XML element can have child elements and attributes. So, simple enough, we want to create an IntelliSense that will give us suggestions for attributes that we can use if we are inside of the XML element tag, and suggestions for child elements if the opening tag is closed. Basically, something like this:

Starting the project

Considering that this post is dedicated to creating the completion provider and not integrating the editor into an application, we’ll use an example from the monaco-editor-samples repository. If you want to follow this post step-by-step, this would be a good time to clone the repository before continuing reading. A sample-editor example is a good start since we already have the code for editor initialization. If you navigate to http://localhost:8888/sample-editor/ in your browser, you should see a simple editor showing some Hello world code.

Step 1 - editor initialization

Since it’s good to separate the js code from the HTML, let’s extract the code that creates the editor into a file called editor.js.

require.config({ paths: { 'vs': '../node_modules/monaco-editor/min/vs' }});

require(['vs/editor/editor.main'], function() {
    var editor = monaco.editor.create(document.getElementById('container'), {
        value: [
            'function x() {',
            '\tconsole.log("Hello world!");',
            '}'
        ].join('\n'),
        language: 'javascript'
    });
});

With this, we just created an editor for JavaScript language. But, since we want to create a completion item provider for XML, we need to edit the code. We’ll change the language property and set the initial value to a part of XML code, so now the editor.js file looks like this:

require.config({paths: { 'vs': '../node_modules/monaco-editor/min/vs' }});

require(['vs/editor/editor.main'], function() {
	var editor = monaco.editor.create(document.getElementById('container'), {
		theme: 'vs-dark', // dark theme
		language: 'xml',
		value: `<?xml version="1.0" encoding="UTF-8"?>\n` // two rows in the initial value
});

Step 2 - `XSD` schema

Monaco editor allows you to make the rules for creating suggestions, but the most natural way to do this for the XML language is by using and XDS schema. To concentrate on the implementation of our provider, let’s include the schema as a string, parse it into an xml element and then keep it in a global variable. We’ll save it in a separate file xml-utils.js, that should be included in the index.html file, just before the editor.js file.

var xmlSchemaString =
`<?xml version="1.0" encoding="UTF-8" ?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="shiporder">
  <xs:annotation>
    <xs:documentation>Details for order shipping.</xs:documentation>
  </xs:annotation>
  <xs:complexType>
    <xs:sequence>
      <xs:element name="orderperson" type="xs:string">
      <xs:annotation>
        <xs:documentation>Person that will handle the order.</xs:documentation>
      </xs:annotation>
      </xs:element>
      <xs:element name="shipto">
        <xs:annotation>
          <xs:documentation>Details of order reciever.</xs:documentation>
        </xs:annotation>
        <xs:complexType>
          <xs:sequence>
            <xs:element name="name" type="xs:string">
              <xs:annotation>
                <xs:documentation>Receiver name.</xs:documentation>
              </xs:annotation>
            </xs:element>
            <xs:element name="address" type="xs:string">
              <xs:annotation>
                <xs:documentation>Receiver address.</xs:documentation>
              </xs:annotation>
            </xs:element>
            <xs:element name="city" type="xs:string">
              <xs:annotation>
                <xs:documentation>Receiver city.</xs:documentation>
              </xs:annotation>
            </xs:element>
            <xs:element name="country" type="xs:string">
              <xs:annotation>
                <xs:documentation>Receiver country.</xs:documentation>
              </xs:annotation>
            </xs:element>
          </xs:sequence>
        </xs:complexType>
      </xs:element>
      <xs:element name="item" maxOccurs="unbounded">
        <xs:annotation>
          <xs:documentation>Order item.</xs:documentation>
        </xs:annotation>
        <xs:complexType>
          <xs:sequence>
            <xs:element name="title" type="xs:string">
              <xs:annotation>
                <xs:documentation>Item title.</xs:documentation>
              </xs:annotation>
            </xs:element>
            <xs:element name="note" type="xs:string" minOccurs="0">
              <xs:annotation>
                <xs:documentation>Item note.</xs:documentation>
              </xs:annotation>
            </xs:element>
            <xs:element name="quantity" type="xs:positiveInteger">
              <xs:annotation>
                <xs:documentation>Quantity of the item.</xs:documentation>
              </xs:annotation>
            </xs:element>
            <xs:element name="price" type="xs:decimal">
              <xs:annotation>
                <xs:documentation>Item price.</xs:documentation>
              </xs:annotation>
            </xs:element>
          </xs:sequence>
        </xs:complexType>
      </xs:element>
    </xs:sequence>
    <xs:attribute name="orderid" type="xs:string" use="required">
      <xs:annotation>
        <xs:documentation>Attribute example.</xs:documentation>
      </xs:annotation>
    </xs:attribute>
  </xs:complexType>
</xs:element>
</xs:schema>`.replace(/xs\:/g, ''); // remove 'xs:' prefix for easier navigation later

function stringToXml(text) {
	var xmlDoc;
	if (window.DOMParser) {
		var parser = new DOMParser();
		xmlDoc = parser.parseFromString(text, 'text/xml');
	}
	else {
		xmlDoc = new ActiveXObject('Microsoft.XMLDOM');
		xmlDoc.async = false;
		xmlDoc.loadXML(text);
	}
	return xmlDoc;
}

var schemaNode = stringToXml(xmlSchemaString).childNodes[0];

The reason that the file is called xml-utils is that it contains an xml function we’ll later use in the implementation of our provider. Now we have all we need to start implementing the completion item provider.

Step 3 - Register the completion item provider

Let’s keep the code for our provider in the completion-provider.js file. For a start, create only one function in it called getXmlCompletionProvider. That function will return the provider we can then register in monaco. We already said that there are one property and two methods that the completion item provider can have. Let’s keep it simple for now, and say that our implementation consists of only provideCompletionItems function, and it always returns an empty array:

function getXmlCompletionProvider(monaco) {
    return {
        providerCompletionItems: function (model, position) {
            return [];
        }
    };
}

model is of type IReadOnlyModel and contains the editor content at the time of evaluating suggestions. If you follow the link above, you will see that IReadOnlyModel has a lot of useful methods on it - and we’ll use some in our implementation, too. The position property is of type Position and, as the type name says, it contains information about the position of the pointer.

Let’s register our dummy provider, so our editor will use it. To do that, we need to add a line of code to the editor.js:

require.config({paths: { 'vs': '../node_modules/monaco-editor/min/vs' }});

require(['vs/editor/editor.main'], function() {
	var editor = monaco.editor.create(document.getElementById('container'), {
		theme: 'vs-dark',
		language: 'xml',
		value: `<?xml version="1.0" encoding="UTF-8"?>\n`
});

// register a completion item provider for xml language
monaco.languages.registerCompletionItemProvider('xml', getXmlCompletionProvider(monaco));

All that’s left now is to actually implement the rules for suggestions.

Step 4 - Provider implementation

Let’s stop for a while to think how we want out provider to work. Our implementation will be similar to this:

function getXmlCompletionProvider(monaco) {
    return {
        provideCompletionItems: function (model, position) {
            // get editor content before the pointer
            let textUntilPosition = getTextBeforePointer();
            // get content info - are we inside of the area where we don't want suggestions,
            // what is the content without those areas
            let info = getAreaInfo(textUntilPosition); // isCompletionAvailable, clearedText
            // if we don't want any suggestions, return empty array
            if (!info.isCompletionAvailable) {
                return [];
            }
            // if we want suggestions, inside of which tag are we?
            var lastTag = getLastOpenedTag(info.clearedText);
            // parse the content (not cleared text) into an xml document
            var xmlDoc = stringToXml(textUntilPosition);
            // get opened tags to see what tag we should look for in the XSD schema
            var openedTags;
            // get the elements/attributes that are already mentioned in the element we're in
            var usedItems;
            // find the last opened tag in the schema to see what elements/attributes it can have
            var currentItem;

            // return available elements/attributes if the tag exists in the schema or an empty
            // array if it doesn't
        }
    };
}

We can use IReadOnlyModel’s method getValueInRange to get all the editor’s content before the pointer position. To do that, we need to tell our function what’s our start and end position. We want it to start at the beginning of the content, and we have the end information in the position variable, so we can replace the getTextBeforePointer() call with the following:

model.getValueInRange({startLineNumber: 1, startColumn: 1, endLineNumber: position.lineNumber, endColumn: position.column})

The next step is a bit more complicated than just a function call. We will actually need to implement our function getAreaInfo. We want it to tell us if completion should be available, and we’ll also get it to return the content without parts where the completion is not available. The reason for the latter is that we’ll use cleared text while looking for the last opened tag. We could use the original content, but then we’d (again) have to watch if the last tag we found is in one of the areas that don’t matter for this (e.g. in a comment). Let’s take a look at the function’s implementation:

function getAreaInfo(text) {
	// opening for strings, comments and CDATA
	var items = ['"', '\'', '<!--', '<![CDATA['];
	var isCompletionAvailable = true;
	// remove all comments, strings and CDATA
	text = text.replace(/"([^"\\]*(\\.[^"\\]*)*)"|\'([^\'\\]*(\\.[^\'\\]*)*)\'|<!--([\s\S])*?-->|<!\[CDATA\[(.*?)\]\]>/g, '');
	for (var i = 0; i < items.length; i++) {
		var itemIdx = text.indexOf(items[i]);
		if (itemIdx > -1) {
			// we are inside one of unavailable areas, so we remove that area
			// from our clear text
			text = text.substring(0, itemIdx);
			// and the completion is not available
			isCompletionAvailable = false;
		}
	}
	return {
		isCompletionAvailable: isCompletionAvailable,
		clearedText: text
	};
}

Using this we can determine if we should look for the completion items or not, and we have only one more step before consulting our XSD schema to find items if we need them. We don’t want to return everything we have in our schema. For example, we don’t want to suggest item’s title if our pointer is inside of the shipto element, so let’s find what the last opened element was:

function getLastOpenedTag(text) {
	// get all tags inside of the content
	var tags = text.match(/<\/*(?=\S*)([a-zA-Z-]+)/g);
	if (!tags) {
		return undefined;
	}
	// we need to know which tags are closed
	var closingTags = [];
	for (var i = tags.length - 1; i >= 0; i--) {
		if (tags[i].indexOf('</') === 0) {
			closingTags.push(tags[i].substring('</'.length));
		}
		else {
			// get the last position of the tag
			var tagPosition = text.lastIndexOf(tags[i]);
			var tag = tags[i].substring('<'.length);
			var closingBracketIdx = text.indexOf('/>', tagPosition);
			// if the tag wasn't closed
			if (closingBracketIdx === -1) {
				// if there are no closing tags or the current tag wasn't closed
				if (!closingTags.length || closingTags[closingTags.length - 1] !== tag) {
					// we found our tag, but let's get the information if we are looking for
					// a child element or an attribute
					text = text.substring(tagPosition);
					return {
						tagName: tag,
						isAttributeSearch: text.indexOf('<') > text.indexOf('>')
					};
				}
				// remove the last closed tag
				closingTags.splice(closingTags.length - 1, 1);
			}
			// remove the last checked tag and continue processing the rest of the content
			text = text.substring(0, tagPosition);
		}
	}
}

Now we’re ready to search for the completion item suggestions. We need to find the opened tags sequence to see what item from schema we need. The reason for this is that we could have two elements that can have a child element with the same name. For example, we’d need this if item’s element title was called name (because shipto can have an element called name, too), so we wouldn’t have sufficient information with just the name of the last opened tag. We also need to find the used items to know which of the child elements should not be available at our cursor position. We need to add this to our getXmlCompletionProvider function:

// get opened tags to see what tag we should look for in the XSD schema
var openedTags = [];
// get the elements/attributes that are already mentioned in the element we're in
var usedItems = [];
var isAttributeSearch = lastOpenedTag && lastOpenedTag.isAttributeSearch;
// no need to calculate the position in the XSD schema if we are in the root element
if (lastOpenedTag) {
  // parse the content (not cleared text) into an xml document
  var xmlDoc = stringToXml(textUntilPosition);
  var lastChild = xmlDoc.lastElementChild;
  while (lastChild) {
    openedTags.push(lastChild.tagName);
    // if we found our last opened tag
    if (lastChild.tagName === lastOpenedTag.tagName) {
      // if we are looking for attributes, then used items should
      // be the attributes we already used
      if (lastOpenedTag.isAttributeSearch) {
        var attrs = lastChild.attributes;
        for (var i = 0; i< attrs.length; i++) {
          usedItems.push(attrs[i].nodeName);
        }
      }
      else {
        // if we are looking for child elements, then used items
        // should be the elements that were already used
        var children = lastChild.children;
        for (var i = 0; i < children.length; i++) {
          usedItems.push(children[i].tagName);
        }
      }
      break;
    }
    // we haven't found the last opened tag yet, so we move to
    // the next element
    lastChild = lastChild.lastElementChild;
  }
}

We still haven’t found our last tag in the XSD schema. We’ll use the information about opened tags to get the element we need. Add these lines after the code written above:

// find the last opened tag in the schema to see what elements/attributes it can have
var currentItem = schemaNode;
for (var i = 0; i < openedTags.length; i++) {
  if (currentItem) {
    currentItem = findElements(currentItem.children, openedTags[i]);
  }
}

The logic for finding the elements is a bit complex, so we’ll implement it in a new method findElements that accepts element’s children and tag name as arguments.

function findElements(elements, elementName) {
	for (var i = 0; i < elements.length; i++) {
		// we are looking for elements, so we don't need to process annotations and attributes
		if (elements[i].tagName !== 'annotation' && elements[i].tagName !== 'attribute') {
			// if it is one of the nodes that do not have the info we need, skip it
			// and process that node's child items
			if (shouldSkipLevel(elements[i].tagName)) {
				var child = findElements(elements[i].children, elementName);
				// if child exists, return it
				if (child) {
					return child;
				}
			}
			// if there is no elementName, return all elements (we'll need this later)
			// this is for the case when we want elements, but we need to remove the
      // 'complexType', 'sequence' and 'all' tags
			else if (!elementName) {
				return elements;
			}
			// find all the element attributes, and if it's name is the same
			// as the element we're looking for, return the element.
			else if (getElementAttributes(elements[i]).name === elementName) {
				return elements[i];
			}
		}
	}
}

The implementation of the shouldSkipLevel function is self-explanatory:

function shouldSkipLevel(tagName) {
	// if we look at the XSD schema, these nodes are containers for elements,
	// so we can skip that level
	return tagName === 'complexType' || tagName === 'all' || tagName === 'sequence';
}

As for the getElementAttributes function, keep in mind that we are processing our XSD schema as an XML, so an example of the element would be:

<xs:element name="note" type="xs:string" minOccurs="0" />

Now, the XML element note has the name attribute value note in the XSD. That is why we need to check if the element has the attribute name that has the same value as the opened tag. The getElementAttributes is actually just returning all of the element’s attributes in an object:

function getElementAttributes(element) {
	var attrs = {};
	for (var i = 0; i < element.attributes.length; i++) {
		attrs[element.attributes[i].name] = element.attributes[i].value;
	}
	// return all attributes as an object
	return attrs;
}

And the final part of our implementation is to get available suggestions for the tag that was opened last. We have two scenarios

we are looking for element attributes
we are looking for element’s children

This is the last part of our getXmlCompletionProvider function:

if (lastOpenedTag.isAttributeSearch) {
  // get attributes completions
  return currentItem ? getAvailableAttribute(monaco, currentItem.children, usedItems) : [];
}
else {
  // get elements completions
  return currentItem ? getAvailableElements(monaco, currentItem.children, usedItems) : [];
}

We’ll use the same information in both cases. Our methods will work with child elements from the currentItem (XSD node for our last opened tag) and usedItems array.

Find attribute suggestions

We’ll go through all the attribute elements from the XSD schema:

function getAvailableAttribute(monaco, elements, usedChildTags) {
	var availableItems = [];
	var children;
	for (var i = 0; i < elements.length; i++) {
		// annotation element only contains documentation,
		// so no need to process it here
		if (elements[i].tagName !== 'annotation') {
			// get all child elements that have 'attribute' tag
			children = findAttributes([elements[i]])
		}
	}
	// if there are no attributes, then there are no
	// suggestions available
	if (!children) {
		return [];
	}
	for (var i = 0; i < children.length; i++) {
		// get all attributes for the element
		var attrs = getElementAttributes(children[i]);
		// accept it in a suggestion list only if it is available
		if (isItemAvailable(attrs.name, attrs.maxOccurs, usedChildTags)) {
			// mark it as a 'property', and get the documentation
			availableItems.push({
				label: attrs.name,
				kind: monaco.languages.CompletionItemKind.Property,
				detail: attrs.type,
				documentation: getItemDocumentation(children[i])
			});
		}
	}
	// return the elements we found
	return availableItems;
}

Firstly, we’re using findAttributes function to get all those elements:

function findAttributes(elements) {
	var attrs = [];
	for (var i = 0; i < elements.length; i++) {
		// skip level if it is a 'complexType' tag
		if (elements[i].tagName === 'complexType') {
			var child = findAttributes(elements[i].children);
			if (child) {
				return child;
			}
		}
		// we need only those XSD elements that have a
		// tag 'attribute'
		else if (elements[i].tagName === 'attribute') {
			attrs.push(elements[i]);
		}
	}
	return attrs;
}

Let’s see the implementations of other functions we used here, and then we’ll explain the construction of suggestion items.

function isItemAvailable(itemName, maxOccurs, items) {
	// the default for 'maxOccurs' is 1
	maxOccurs = maxOccurs || '1';
	// the element can appear infinite times, so it is available
	if (maxOccurs && maxOccurs === 'unbounded') {
		return true;
	}
	// count how many times the element appeared
	var count = 0;
	for (var i = 0; i < items.length; i++) {
		if (items[i] === itemName) {
			count++;
		}
	}
	// if it didn't appear yet, or it can appear again, then it
	// is available, otherwise it't not
	return count === 0 || parseInt(maxOccurs) > count;
}

function getItemDocumentation(element) {
	for (var i = 0; i < element.children.length; i++) {
		// annotation contains documentation, so calculate the
		// documentation from it's child elements
		if (element.children[i].tagName === 'annotation') {
			return getItemDocumentation(element.children[0]);
		}
		// if it's the documentation element, just get the value
		else if (element.children[i].tagName === 'documentation') {
			return element.children[i].textContent;
		}
	}
}

We provide four properties to our completion items:

label - the tag/attribute that will be inserted
kind - what kind of suggestion is it (in this post, we only used Property and Field, but there are many others)
detail - gives a short detail information about the suggestion; we used it to display the element/attribute value type
documentation - could be a description of the suggestion

The editor is now providing us the attribute suggestions based on the XSD schema we used.

Find child element suggestions

Search for child elements is a lot like searching for the attributes. We use the functions already implemented in steps before this one:

function getAvailableElements(monaco, elements, usedItems) {
	var availableItems = [];
	var children;
	for (var i = 0; i < elements.length; i++) {
		// annotation element only contains documentation,
		// so no need to process it here
		if (elements[i].tagName !== 'annotation') {
			// get all child elements that have 'element' tag
			children = findElements([elements[i]])
		}
	}
	// if there are no such elements, then there are no suggestions
	if (!children) {
		return [];
	}
	for (var i = 0; i < children.length; i++) {
		// get all element attributes
		let elementAttrs = getElementAttributes(children[i]);
		// the element is a suggestion if it's available
		if (isItemAvailable(elementAttrs.name, elementAttrs.maxOccurs, usedItems)) {
			// mark it as a 'field', and get the documentation
			availableItems.push({
				label: elementAttrs.name,
				kind: monaco.languages.CompletionItemKind.Field,
				detail: elementAttrs.type,
				documentation: getItemDocumentation(children[i])
			});
		}
	}
	// return the suggestions we found
	return availableItems;
}

Now we can see the suggestion for child elements. Each element has the type information in upper right corner and the documentation below the label. If you look at the little icon left to the suggestion label, you can also see the difference between attributes and child elements.

We now have our own completion items suggested to us while using the editor, but let’s add a little detail to it. At the moment, we need to explicitly ask for suggestions. But, we can assume that we’d like to get suggestions every time we open a tag. So let’s tell our editor to do that for us. In the editor.js add suggestOnTriggerCharacters: true to editor initialization, and in the custom-provider.js add triggerCharacters: ['<'] to the object we return in the getXmlCompletionProvider function.

Summary

We have implemented our custom completion item provider and improved the XML IntelliSense in Monaco Editor. If you’re interested, you can find the code here. Of course, there is room for improvement. For example, we could take into account the whole word at the position we’re at, or go through the items that appear after the position to see if we should suggest an item or not. But I believe this is a good start.