Data integration On-premise Custom

Custom on-premise data integration development

Appfront builds data integration that runs entirely within your own network. Systems that don't know each other are connected, data is converted and checked along the way, and you have visibility of what was passed on and when, without your data passing through a cloud service. Get in touch to discuss what your situation requires.

What is on-premise data integration?

On-premise data integration is the connecting of systems where all processing stays within your own infrastructure: on your servers, in your data centre or in a private cloud that you manage. The integrations themselves are not fundamentally different from cloud integration (retrieve, convert, validate, write), but where it happens is, and that is precisely why organisations choose it.

The reason is usually one of three things. There is a legal or contractual requirement that data must not leave the network. There are systems that simply can't be reached from outside, such as production control or a legacy package without an API. Or the data volumes are so large that sending them back and forth to a cloud service is impractical.

Appfront builds these integrations as custom software, including monitoring and error handling, so that maintenance doesn't come down to someone checking every morning whether the overnight script has run. If you mainly want advice on your data landscape, see data integration consulting.

Your data stays in

All processing runs on infrastructure that you manage. No stopover at an external service, no discussion with your security officer about data processing agreements.

Systems without an API too

Database integrations, file exchange, queues and, where necessary, a wrapper around a legacy package, because reality rarely consists of neat REST APIs.

Visibility of what happens

Logging, reconciliation and alerting per integration, so that a failed run doesn't only come to light when a customer calls to say an order is missing.

On-premise or cloud? Where to draw the line

The choice is rarely a matter of principle and is usually practical. This page covers the situation where on-premise is necessary or sensible; if it isn't, a cloud solution is often simpler.

This page

Within your own walls

There is a requirement that data must not leave the network, or there are source systems that can't be reached from outside, or the volumes make exchange with a cloud service impractical. The integrations run on your own infrastructure.

Related

Advice or AI within your own environment

If you first want to know what your data landscape should look like, see data integration consulting. If the same sovereignty question applies to AI, a private on-premise LLM is the relevant page.

Our development process for your integration landscape

Integration projects get stuck on assumptions about what a source system can do. We test that early, in the real environment.

1
Discovery & analysis

We map out which systems need to be integrated, what they allow technically, who owns them and exactly which data has to flow between them. Where possible, we connect to a test environment early on, because the documentation for a legacy package is rarely entirely accurate.

2
Design

We design the integrations, the canonical data model, the transformation and validation rules and the error-handling path, including what should happen when a source system is offline for a day.

3
Build and iteration

We build in short iterations with automated tests, structured logging and monitoring. You see working versions along the way and steer based on what your team actually needs in practice, rather than on a specification written months ago.

4
Go-live & maintenance

A controlled go-live with validation and a safety net, followed by ongoing management, monitoring and further development as your system landscape or data model changes.

What custom on-premise data integration delivers in practice

What you need varies from one landscape to the next. These are the components that come up most often.

Integrations with source systems

Via API, database, file exchange or queue, depending on what the system allows. For packages with no integration options, we build an intermediate layer where necessary.

Transformation and validation

Converting data into a single shared model, with validation against the rules that matter in your organisation, so that errors stop at the boundary instead of seeping through.

Synchronisation and scheduling

Event-driven where possible and scheduled where necessary, with a defined sequence and dependencies between integrations.

Error handling and recovery

Failed messages end up in a recovery queue where they can be inspected and resubmitted, instead of in a log file nobody reads.

Monitoring and alerting

Dashboards per integration showing throughput, latency and error rate, plus alerts to the administrator when a flow stalls or deviates from its normal pattern.

Management by your own team

Documentation, repeatable deployment and a management environment so that your own administrators can monitor, restart and adjust integrations without having to call us in.

Typical use cases in practice

On-premise integration matters most where sensitivity, technology or volume stand in the way of the cloud.

Healthcare

Organisations that exchange patient data between systems and for which the requirement that data never leaves the organisation outweighs the convenience of a cloud service.

Government and semi-public bodies

Organisations with data sovereignty requirements, and with base registers and case management systems that can only be reached from within their own network.

Industry and manufacturing

Integrations between production control and office automation, where the control layer is kept separate from the internet for safety reasons.

Financial services

Organisations with contractual or regulatory requirements about where data may be processed, and with core systems that cannot be opened up through a public integration.

Technology we use

We prefer to work with components that you can run and understand yourself, rather than an integration platform that becomes a specialism in its own right. The exact choice depends on your processes, the systems to be integrated and your hosting preferences. We deliberately choose a stack that your own team can manage and develop further, without dependence on per-user licences.

Node.js / Python / .NET PostgreSQL / MySQL / SQL Server Message queues (RabbitMQ, Kafka) Docker / Kubernetes REST, SOAP & file-based integrations Monitoring and alerting

Why choose Appfront for your on-premise integration?

Appfront builds custom software for a wide range of organisations in the Netherlands. With integration, we start by connecting to the real source systems, not with a design on paper. The documentation for a twenty-year-old package is rarely accurate, and you want to know that before a plan is set.

We build with components your own administrators know, so you don't become dependent on an integration platform with its own licences and its own specialists. That is also what makes it possible to move to the cloud later if your requirements change.

On every project we write clear documentation and make sure your own team, or a future supplier, can understand and manage the system. No black box, but transparent code and clear agreements on monitoring, alerting and maintenance. You own the solution and pay no per-user licence.

Also take a look at our broader services around custom software, data integration consulting and private on-premise LLMs. Not sure about the approach? Get in touch.

Security in your integration landscape

An integration layer, by definition, has access to several systems at once, which makes it an attractive target. Appfront builds to the OWASP ASVS, with least-privilege service accounts and encryption even inside your own network, because internal traffic is not automatically safe traffic.

For each integration, we document which data flows, for what purpose and on what legal basis, so your record of processing activities is accurate and GDPR compliance can be demonstrated. Where possible, we limit the flow to the fields that are actually needed rather than passing on entire records.

More about our approach to security: information security policy and CVD policy.

  • Encryption in transit (TLS 1.2+) and at rest
  • Role-based access following least privilege
  • Audit trail of views and changes
  • Secrets in a secure vault, not in code
  • Documented data flows for your processing register
  • Service accounts with minimal permissions per source system

Frequently asked questions about on-premise data integration

Answers to the questions we get asked most often.

Connecting systems where all processing runs on infrastructure you manage yourself: your servers, your data centre or a private cloud. Data is retrieved, transformed, validated and written back without passing through an external service. Functionally it is the same as cloud integration; the difference is where it runs and who can access it.

It is necessary when there is a legal or contractual requirement that data must not leave the network, when source systems cannot be reached from outside, or when the volumes make exchange with a cloud service impractical. If none of these three applies, a cloud solution is usually easier to manage, and we will tell you so.

Yes, and in practice that is often the case. We then work with a direct database integration, scheduled file exchange or an intermediate layer around the package. What matters is that we first test what is possible in the real environment, and whether the vendor of that package has any objections. That last point is a contractual question you do not want to run into afterwards.

Failed messages go into a recovery queue together with the reason for the failure and the original content, so they can be resubmitted after a correction. The administrator receives a notification when a flow stalls or the error rate rises. What exactly should happen when a source system goes down is agreed per integration. Sometimes waiting is actually better than carrying on with outdated data.

Preferably your own operations team. We provide documentation, a repeatable deployment and a management environment in which integrations can be monitored, restarted and adjusted. If you would like us to handle the management, that is possible too, but we do not build in a way that forces you to.

Yes, provided we take that into account in the design. By building the integrations as separate components with a clear separation between logic and infrastructure, moving to a cloud environment later is a matter of deployment rather than a rebuild. That is also a reason not to use an integration platform that is tied to a single environment.

Internal traffic is not automatically safe traffic. We encrypt within the network as well, work with least-privilege service accounts, keep secrets in a vault and make sure the integration layer has no more permissions on source systems than strictly necessary for the data it moves.

For a landscape with many off-the-shelf packages that already know each other, an integration platform can be efficient. The drawback is that it becomes a specialism of its own, with licences and expertise you have to buy in. Custom pays off with a limited number of integrations, with unusual source systems, or when you want your own developers to be able to maintain it with the knowledge they already have.

Ready to build your on-premise data integration?

Tell us which systems need to be integrated, why the data has to stay in-house and what is currently done manually or via a script — we're happy to help you think through scope, integrations and the first version. A no-obligation first conversation gives you a clear picture of the possibilities and whether custom software pays off in your situation.

Is this topic relevant within your own organisation too? On applatenmaken.com you can read more about building data integration software.

Edit content