Syntax Coloring for TypeScript keywords + ES7 decorators

[sdargutev]

Currently syntax coloring does not take into consideration Typescript keywords... Adding the following method to the class ts.eclipse.ide.jsdt.internal.ui.editor.TypeScriptSourceViewerConfiguration

@Overridde
public IPresentationReconciler getPresentationReconciler(ISourceViewer sourceViewer) {
        JavaPresentationReconciler presentationReconciler = (JavaPresentationReconciler) super.getPresentationReconciler(sourceViewer);
        DefaultDamagerRepairer dr = new DefaultDamagerRepairer(new JavaScriptCodeScanner(getColorManager(), fPreferenceStore));
        presentationReconciler.setDamager(dr, IDocument.DEFAULT_CONTENT_TYPE);
        presentationReconciler.setRepairer(dr, IDocument.DEFAULT_CONTENT_TYPE);
        return presentationReconciler;
    }

and then including the following class will provide the editor with syntax coloring that will be configurable using the Javascript preferences page. The class is a clean modification of org.eclipse.jdt.internal.ui.text.java.JavaCodeScanner

package ts.eclipse.ide.jsdt.internal.ui.editor;

/*******************************************************************************
 * Copyright (c) 2000, 2011 IBM Corporation and others.
 * All rights reserved. This program and the accompanying materials
 * are made available under the terms of the Eclipse Public License v1.0
 * which accompanies this distribution, and is available at
 * http://www.eclipse.org/legal/epl-v10.html
 *
 *******************************************************************************/

import java.util.ArrayList;
import java.util.List;

import org.eclipse.jface.preference.IPreferenceStore;
import org.eclipse.jface.text.rules.ICharacterScanner;
import org.eclipse.jface.text.rules.IRule;
import org.eclipse.jface.text.rules.IToken;
import org.eclipse.jface.text.rules.SingleLineRule;
import org.eclipse.jface.text.rules.Token;
import org.eclipse.jface.text.rules.WhitespaceRule;
import org.eclipse.jface.util.PropertyChangeEvent;
import org.eclipse.wst.jsdt.core.JavaScriptCore;
import org.eclipse.wst.jsdt.internal.ui.text.AbstractJavaScanner;
import org.eclipse.wst.jsdt.internal.ui.text.CombinedWordRule;
import org.eclipse.wst.jsdt.internal.ui.text.JavaWhitespaceDetector;
import org.eclipse.wst.jsdt.internal.ui.text.JavaWordDetector;
import org.eclipse.wst.jsdt.ui.text.IColorManager;
import org.eclipse.wst.jsdt.ui.text.IJavaScriptColorConstants;

public final class JavaScriptCodeScanner extends AbstractJavaScanner {

    private static final class OperatorRule implements IRule {

        private final char[] JAVA_OPERATORS = { ';', '.', '=', '/', '\\', '+', '-', '*', '<', '>', ':', '?', '!', ',',
                '|', '&', '^', '%', '~' };

        private final IToken fToken;

        public OperatorRule(IToken token) {
            fToken = token;
        }

        public boolean isOperator(char character) {
            for (int index = 0; index < JAVA_OPERATORS.length; index++) {
                if (JAVA_OPERATORS[index] == character)
                    return true;
            }
            return false;
        }

        @Override
        public IToken evaluate(ICharacterScanner scanner) {

            int character = scanner.read();
            if (isOperator((char) character)) {
                do {
                    character = scanner.read();
                } while (isOperator((char) character));
                scanner.unread();
                return fToken;
            } else {
                scanner.unread();
                return Token.UNDEFINED;
            }
        }
    }

    private static final class BracketRule implements IRule {

        private final char[] JAVA_BRACKETS = { '(', ')', '{', '}', '[', ']' };
        private final IToken fToken;

        public BracketRule(IToken token) {
            fToken = token;
        }

        public boolean isBracket(char character) {
            for (int index = 0; index < JAVA_BRACKETS.length; index++) {
                if (JAVA_BRACKETS[index] == character)
                    return true;
            }
            return false;
        }

        @Override
        public IToken evaluate(ICharacterScanner scanner) {

            int character = scanner.read();
            if (isBracket((char) character)) {
                do {
                    character = scanner.read();
                } while (isBracket((char) character));
                scanner.unread();
                return fToken;
            } else {
                scanner.unread();
                return Token.UNDEFINED;
            }
        }
    }

    private static final String SOURCE_VERSION = JavaScriptCore.COMPILER_SOURCE;

    static String[] fgKeywords = { "break", "as", "any", "case", "implements", "boolean", "catch", "interface",
            "constructor", "class", "let", "declare", "const", "package", "get", "continue", "private", "module",
            "debugger", "protected", "require", "default", "public", "number", "delete", "static", "set", "do", "yield",
            "string", "else", "symbol", "enum", "type", "export", "from", "extends", "of", "false", "finally", "for",
            "function", "if", "import", "in", "instanceof", "new", "null", "super", "switch", "this", "throw", "true",
            "try", "typeof", "var", "void", "while", "with" };

    private static final String RETURN = "return";

    private static String[] fgTokenProperties = { IJavaScriptColorConstants.JAVA_KEYWORD,
            IJavaScriptColorConstants.JAVA_STRING, IJavaScriptColorConstants.JAVA_DEFAULT,
            IJavaScriptColorConstants.JAVA_KEYWORD_RETURN, IJavaScriptColorConstants.JAVA_OPERATOR,
            IJavaScriptColorConstants.JAVA_BRACKET, };


    public JavaScriptCodeScanner(IColorManager manager, IPreferenceStore store) {
        super(manager, store);
        initialize();
    }

    @Override
    protected String[] getTokenProperties() {
        return fgTokenProperties;
    }

    @Override
    protected List<IRule> createRules() {

        List<IRule> rules = new ArrayList<IRule>();

        // Add rule for character constants.
        Token token = getToken(IJavaScriptColorConstants.JAVA_STRING);
        rules.add(new SingleLineRule("'", "'", token, '\\')); //$NON-NLS-2$ //$NON-NLS-1$

        Token defaultToken = getToken(IJavaScriptColorConstants.JAVA_DEFAULT);

        // Add generic whitespace rule.
        rules.add(new WhitespaceRule(new JavaWhitespaceDetector(), defaultToken));

        String version = getPreferenceStore().getString(SOURCE_VERSION);

        // Add word rule for new keywords, see bug 4077
        JavaWordDetector wordDetector = new JavaWordDetector();
        CombinedWordRule combinedWordRule = new CombinedWordRule(wordDetector, defaultToken);

        // Add rule for operators
        token = getToken(IJavaScriptColorConstants.JAVA_OPERATOR);
        rules.add(new OperatorRule(token));

        // Add rule for brackets
        token = getToken(IJavaScriptColorConstants.JAVA_BRACKET);
        rules.add(new BracketRule(token));

        // Add word rule for keyword 'return'.
        CombinedWordRule.WordMatcher returnWordRule = new CombinedWordRule.WordMatcher();
        token = getToken(IJavaScriptColorConstants.JAVA_KEYWORD_RETURN);
        returnWordRule.addWord(RETURN, token);
        combinedWordRule.addWordMatcher(returnWordRule);

        // Add word rule for keywords, types, and constants.
        CombinedWordRule.WordMatcher wordRule = new CombinedWordRule.WordMatcher();
        token = getToken(IJavaScriptColorConstants.JAVA_KEYWORD);
        for (int i = 0; i < fgKeywords.length; i++)
            wordRule.addWord(fgKeywords[i], token);

        combinedWordRule.addWordMatcher(wordRule);

        rules.add(combinedWordRule);

        setDefaultReturnToken(defaultToken);
        return rules;
    }

    @Override
    public boolean affectsBehavior(PropertyChangeEvent event) {
        return event.getProperty().equals(SOURCE_VERSION) || super.affectsBehavior(event);
    }
}

[angelozerr]

Thanks @sdargutev for your suggestion. I'm working on JSX Syntax Coloration #11

When I will finish my work, I will study your suggestion. I would prefer extends JavaCodeScanner and add word rule instead of copy/paste JavaCodeScanner (not sure if it's possible).

[sdargutev]

I tried that but JavaCodeScanner is final...
btw. good work with this project so far. hope you keep it going!

[angelozerr]

I tried that but JavaCodeScanner is final...

Perhaps with delegate pattern? I don't know....

btw. good work with this project so far. hope you keep it going!

Thanks!

[angelozerr]

@sdargutev could you just tell me which keywords are missing.

I will try to integrate your idea + support for ES7 decorators.

Thanks!

[sdargutev]

following keywords are missing in org.eclipse.jdt.internal.ui.text.java.JavaCodeScanner. I think delegation pattern wont work because the methods are protected. Unless of course you do things with generics or naming the package 'org.eclipse.jdt.internal.ui.text.java' .... or there is some trick that i dont know about. You can see the complete list of Typescript keywords in my first post above

"as", "any","constructor", "let", "declare", "get", "module", "debugger", "require", "number", "delete", "set", "yield", "string", "symbol", "type", "export", "from", "of", "function", "in", "typeof", "var", "with"

waiting for 1.1.0 :)

[angelozerr]

Thanks @sdargutev.

You can install 1.1.0 SNAPSHOT (see https://github.com/angelozerr/typescript.java/wiki/Installation-Update-Site)

Here the result:

See https://github.com/angelozerr/typescript.java/wiki/New-and-Noteworthy-1.1.0#typescript-editor

[sdargutev]

Looks nice! Thanks! @angelozerr