Follow our Hackathon live on YouTube

https://youtu.be/ZMbUzb27amo

Agenda

Friday

12:00-12:45 - Welcome

12:45-13:40 - Task introduction & group finding

13:40 - Start Hackathon

14:45-15:15 - Open Question, Intro Bonus task

15:15-15:45 - Intro into Azure

19:00 - Virtual dinner

...hack … hack … hack

Saturday

09:00 - 10:00 - Fun activity: Yoga & Energizers

10:30-11:15 - In-depth intro Bonus task

16:00 - Deadline - Submission Task

16:00-17:00 - Preparation time pitch

17:00-19:00 - 5-minute pitch for jury

19:10-19:30 - Announcement winners

19:30 - Virtual Apero

 

Background: FDFA IT Support Mailbot

The Swiss Federal Department of Foreign Affairs (FDFA) (Eidgenössisches Departement für auswärtige Angelegenheiten, EDA) determines and coordinates Swiss foreign policy on behalf of the Federal Council. The FDFA consists of the organizational units at head office in Bern and the network of Swiss representations, which includes embassies, consulates, cooperation offices and missions.

The FDFA internal IT Helpdesk

FDFA has its own IT department, which provides IT services. The IT Helpdesk is in charge for supporting internal users. The Helpdesk collaborators treat around 95 phone calls and 80 incoming mails per day in various languages (DE, FR, IT, EN). About 40’000 incident tickets are opened every year.

Due to the high workload support mails are sometimes only treated after several hours, which can result in users calling in and creating even more workload.

Once the Helpdesk collaborator has analyzed a support request he creates an incident in the IT Service Management Tool. He assigns the incident to a service (e.g. Mobile Communication), to a support group (the group that will treat the incident) and evaluates the urgency und impact of the request (which determines the priority).

All interactions and activities are recorded in the ticketing system.

A “Mailbot” for the FDFA IT Helpdesk

To offer a better support service to the users, the FDFA IT Helpdesk plans to use a Mailbot (for German mails only). The Mailbot should analyze incoming support mails with a pretrained AI model, open an incident in the ticketing system with the correct service category and send a receipt mail to the user. This receipt mail should also contain links to manuals from a collection of how-to manuals, that the Helpdesk team maintains. Helpdesk employees will then take over and correct & finalize the created case.

«PoCathon» Challenge: Create the best ML Model for the Mailbot

A PoCathon is a proof-of-concept (PoC) in the form of a hackathon.

FDFA wants to discover what approach performs best to analyze and classify received Mails by the internal IT Helpdesk.

Every Team creates its own PoC and will compete against the other AI solutions.

Your Objective

Your objective today is:
Based on real IT helpdesk support mails, train one or multiple machine learning (ML) models that predict the following attributes as reliable as possible (measured by F1 Score):

[Task 1] Predict Labels IT-Service & How-To Manuals
  1. Which service is the incident about? (e.g. “S_BA_Account” or “S_BA_Mailbox”). Update: Predict all Services 1:1 as in the training data and do NOT use an aggregated EDA_OTHERS category, as was wrongly stated in the Hackathon Intro.

  2. Which manuals of the provided how-to manual list would fit? Try to find max. 4 matches per incoming E-Mail.

[Task 2] Service-Merging Strategy

Find a solution for the evaluation of the best possible services to merge with each other into a single service.

Try to come up with ready-to-use code that benchmarks different combinations of 2 or more services and show the resulting differences. Try to run automatic tests of one or more merges in parallel. 

You can use any programming language.

[Task 3] Bonus Challenge: Clustering to find new Manuals

If you completed the first parts of the PoC you may try to solve the following question:

Which manuals are missing on the list? Try to perform a clustering of the E-Mails, so that new manual categories can be identified and the Helpdesk team can create such how-to manuals.

Input Data: Support Mails (cf. 2.2 Data)

Hackathon Sponsors

Prizes

Flight with a PC-12

Bodyflying Voucher

Adventure Room Voucher

Devpost Achievements

Submitting to this hackathon could earn you:

Requirements

Submission Details

Below you can find what kind of data you need to submit and how to submit it.

We need only one Submission per Team. Please use your Hackathon Teamname in DevPost and in Kaggle for the submissions.

You can only submit one file and multiple additional links per team. This means, you need to put all your documents in one zip file before uploading. Make sure the naming reflects the solved task (e.g. task1_readme.pdf)

Task 1
What to submit Where to submit
ML Model(s) in a GitHub/GitLab/Bitbucket repo for classification of Support Mail Requests (Service) and the assignment of how-to manuals (use only the list of manuals provided; no self-identified manuals). As a link in your DevPost Submission
A One-Pager PDF or README File describing your chosen strategy and technologies for this task. As a file in the zip for your DevPost Submission
Your predictions of the sample data as a csv File submitted to Kaggle.

As a CSV file in your Kaggle Submissions. Use your Hackathon Teamname as the Kaggle Teamname. Use ","-separated format (not ";") and do NOT predict an EDA_OTHERS aggregated label; instead 1:1 as in the training set.

--> A Sample Submission File is available on Kaggle under the Tab "Data". Of course your file should have the exact amount of rows and all Ids in your TEST_reduced file inside the ZIP.

Update to Manuals:
When predicting multiple Manuals per Row, please use the same separator as used in the TRAIN dataset (that is "|") between multiple manuals for the same row. Order: You MUST sort these alphabetically, so that kaggle can match your output.

Task 2
What to submit Where to submit
Running code in a GitHub/GitLab/Bitbucket Repo that goes through training of an ML Model various times with different combinations of Service labels. As a link in your DevPost Submission
Describe in a Readme File or One-Pager PDF what exactly your code does and your chosen strategy and technologies. Also try to argue, why your strategy produces meaningful results (we do not provide “hard measures” here). As a file in the zip for your DevPost Submission
Task 3
What to submit Where to submit
If existent: Model as code on Github/GitLab/Bitbucket or prose text algorithm/procedure to identify additional how-to manuals, that the Helpdesk Team should identify. Ideally also a list of such manuals, which you were able to identify this way (Prose Text can be combined with the following One-Pager). As a link in your DevPost Submission
A One-Pager PDF or README File describing your chosen strategy and technologies for this task. As a file in the zip for your DevPost Submission
A list of how-to manuals you identified (this list can include the existing FDFA manuals list, or preferably even be fully independent) As a file in the zip for your DevPost Submission

 

Judges

Tomaso Bezzola

Tomaso Bezzola
FDFA

Patrick Zwicker

Patrick Zwicker
FDFA

Steve Andrey

Steve Andrey
FDFA

Mark Bosshard

Mark Bosshard
ipt

Valentin Trifonov

Valentin Trifonov
ipt

Oliver Richter

Oliver Richter
eestec

Selim Naji

Selim Naji
eestec

Judging Criteria

  • [Task 1] F1 Score
    For each category "IT-Services" and "How-To Manuals" the team with the best F1 score (Micro F1-Score) gets 7.5 points. The other teams get points proportional to the best F1 score. A maximum of 15 points can be achieved.
  • [Task 1] Tools & Methodology
    The jury evaluates the choice of tools and methodology for solving the challenge. This can yield an additional 5 points.
  • [Task 2] Architecture, Methodology, Output
    The jury evaluates the choice of tools, methodology and output for solving the challenge. The originality of your strategy and the meaningfulness of your service-merging strategy will be rated. A maximum of 15 points can be achieved.
  • [Task 3] Originality of Clustering
    The jury evaluates the originality of your strategy and the meaningfulness of your automatically generated categories. This part may yield up to an additional 10 points.
  • Presentation
    The quality, timing and originality of your final presentation will be assessed by the jury. This can yield an additional 5 points.

Tell your friends

Hackathon sponsors

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.